ConfD User Guide 6.6

confd_user_guide-6.6

User Manual:
Open the PDF directly: View PDF .
Page Count: 1026
Download
Open PDF In Browser	View PDF
ConfD User Guide
Copyright © 2005-2018 Tail-f Systems
ConfD 6.6
March 2, 2018

ConfD User Guide
ConfD 6.6
Publication date March 2, 2018
Copyright © 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018 Tail-f Systems
All contents in this document are confidential and proprietary to Tail-f Systems.

Table of Contents
1. About the Documentation ................................................................................................. 1
1.1. How to Read This Guide ........................................................................................ 1
1.2. Getting Documentation .......................................................................................... 1
1.3. Formatting Conventions ......................................................................................... 1
1.4. Documentation Feedback ........................................................................................ 2
2. An introduction to ConfD ................................................................................................. 3
2.1. An on-device software system for configuration management ........................................ 3
2.2. ConfD Architecture ............................................................................................... 3
3. The YANG Data Modeling Language ................................................................................. 8
3.1. The YANG Data Modeling Language ....................................................................... 8
3.2. YANG in ConfD .................................................................................................. 8
3.3. YANG Introduction ............................................................................................... 8
3.4. Working With YANG Modules ............................................................................. 16
3.5. Integrity Constraints ............................................................................................. 18
3.6. The when statement ............................................................................................. 20
3.7. Using the Tail-f Extensions with YANG ................................................................. 20
3.8. Custom Help Texts and Error Messages .................................................................. 22
3.9. Hidden Data ....................................................................................................... 24
3.10. An Example: Modeling a List of Interfaces ............................................................ 26
3.11. More on leafrefs ................................................................................................ 34
3.12. Using Multiple Namespaces ................................................................................ 36
3.13. Module Names, Namespaces and Revisions ............................................................ 37
3.14. Hash Values and the id-value Statement ................................................................ 38
3.15. Migrating from Confspecs to YANG ..................................................................... 39
3.16. The pyang tool .................................................................................................. 44
4. Rendering Agents .......................................................................................................... 45
4.1. Introduction ........................................................................................................ 45
4.2. Data Model ........................................................................................................ 45
4.3. Using the CLIs ................................................................................................... 47
4.4. Using NETCONF ................................................................................................ 50
5. CDB - The ConfD XML Database .................................................................................... 53
5.1. Introduction ........................................................................................................ 53
5.2. CDB ................................................................................................................. 53
5.3. An example ........................................................................................................ 54
5.4. Using keypaths ................................................................................................... 58
5.5. A session ........................................................................................................... 59
5.6. CDB subscriptions ............................................................................................... 60
5.7. Reconnect .......................................................................................................... 61
5.8. Loading initial data into CDB ................................................................................ 62
5.9. Automatic schema upgrades and downgrades ........................................................... 63
5.10. Using initialization files for upgrade ..................................................................... 68
5.11. Using MAAPI to modify CDB during upgrade ........................................................ 71
5.12. More complex schema upgrades ........................................................................... 72
5.13. The full dhcpd example ...................................................................................... 75
6. Operational Data ............................................................................................................ 82
6.1. Introduction to Operational Data ............................................................................ 82
6.2. Reading Statistics Data ......................................................................................... 82
6.3. Callpoints and Callbacks ...................................................................................... 83
6.4. Data Callbacks .................................................................................................... 85
6.5. User Sessions and ConfD Transactions .................................................................... 87
6.6. C Example with Operational Data .......................................................................... 87

iii

ConfD User Guide

6.7. The Protocol and a Library Threads Discussion ........................................................ 95
6.8. Operational data in CDB ...................................................................................... 97
6.9. Delayed Replies ................................................................................................ 100
6.10. Caching Operational Data .................................................................................. 101
6.11. Operational data lists without keys ...................................................................... 102
7. The external database API ............................................................................................. 104
7.1. Introduction to external data ................................................................................ 104
7.2. Scenario - The database is a file ........................................................................... 104
7.3. Callpoints and callbacks ...................................................................................... 104
7.4. Data Callbacks .................................................................................................. 105
7.5. User sessions and ConfD Transactions .................................................................. 105
7.6. External configuration data .................................................................................. 108
7.7. External configuration data with transactions .......................................................... 112
7.8. Writable operational data .................................................................................... 116
7.9. Supporting candidate commit ............................................................................... 116
7.10. Discussion - CDB versus external DB ................................................................. 118
8. Configuration Meta-Data ............................................................................................... 120
8.1. Introduction to Configuration Meta-Data ................................................................ 120
8.2. Meta-Data: annotation .................................................................................. 120
8.3. Meta-Data: tag ................................................................................................ 120
8.4. Meta-Data: inactive ...................................................................................... 121
9. Semantic validation ...................................................................................................... 122
9.1. Why Do We Need to Validate ............................................................................. 122
9.2. Syntactic Validation in YANG models .................................................................. 122
9.3. Integrity Constraints in YANG Models .................................................................. 122
9.4. The YANG must Statement ................................................................................. 123
9.5. Validation Logic ................................................................................................ 123
9.6. Validation Points ............................................................................................... 124
9.7. Validating Data in C .......................................................................................... 125
9.8. Validation Points and CDB ................................................................................. 130
9.9. Dependencies - Why Does Validation Points Get Called ........................................... 130
9.10. Configuration Policies ....................................................................................... 132
10. Transformations, Hooks, Hidden Data and Symlinks ......................................................... 134
10.1. Introduction ..................................................................................................... 134
10.2. Transformation Control Flow ............................................................................. 134
10.3. An Example .................................................................................................... 135
10.4. AAA Transform ............................................................................................... 140
10.5. Other Use Cases for Transformations .................................................................. 142
10.6. Hooks ............................................................................................................ 143
10.7. Hidden Data .................................................................................................... 146
10.8. tailf:symlink .................................................................................................... 148
11. Actions ..................................................................................................................... 151
11.1. Introduction ..................................................................................................... 151
11.2. Action as a Callback ........................................................................................ 151
11.3. Action as an Executable .................................................................................... 154
11.4. Related functionality ......................................................................................... 157
12. Notifications .............................................................................................................. 159
12.1. ConfD Asynchronous Events ............................................................................. 159
12.2. Audit Messages ............................................................................................... 160
12.3. Syslog Messages .............................................................................................. 162
12.4. Commit Events ................................................................................................ 162
12.5. Commit Failure Events ..................................................................................... 165
12.6. Confirmed Commit Events ................................................................................ 165
12.7. Commit Progress Events ................................................................................... 165

iv

ConfD User Guide

13.

14.

15.

16.

12.8. User Sessions ..................................................................................................
12.9. High Availability - Cluster Events ......................................................................
12.10. Subagent Events .............................................................................................
12.11. SNMP Agent Audit Log ..................................................................................
12.12. Forwarding Events ..........................................................................................
12.13. In-service Upgrade Events ...............................................................................
12.14. Heartbeat and Health Check Events ...................................................................
12.15. Notification stream Events ...............................................................................
In-service Data Model Upgrade ....................................................................................
13.1. Introduction .....................................................................................................
13.2. Preparing for the Upgrade .................................................................................
13.3. Initializing the Upgrade .....................................................................................
13.4. Performing the Upgrade ....................................................................................
13.5. Committing the Upgrade ...................................................................................
13.6. Aborting the Upgrade .......................................................................................
13.7. Upgrade and HA ..............................................................................................
The AAA infrastructure ...............................................................................................
14.1. The problem ....................................................................................................
14.2. Structure - data models .....................................................................................
14.3. AAA related items in confd.conf ........................................................................
14.4. Authentication .................................................................................................
14.5. Group Membership ...........................................................................................
14.6. Authorization ..................................................................................................
14.7. The AAA cache ...............................................................................................
14.8. Populating AAA using CDB ..............................................................................
14.9. Populating AAA using external data ....................................................................
14.10. Hiding the AAA tree ......................................................................................
The NETCONF Server ................................................................................................
15.1. Introduction .....................................................................................................
15.2. Capabilities .....................................................................................................
15.3. NETCONF Transport Protocols ..........................................................................
15.4. Configuration of the NETCONF Server ...............................................................
15.5. Extending the NETCONF Server ........................................................................
15.6. Monitoring of the NETCONF Server ...................................................................
15.7. Notification Capability ......................................................................................
15.8. Using netconf-console .......................................................................................
15.9. Actions Capability ............................................................................................
15.10. Transactions Capability ...................................................................................
15.11. Proxy Forwarding Capability ............................................................................
15.12. Inactive Capability ..........................................................................................
15.13. Tail-f Identification Capability ..........................................................................
15.14. The Query API ..............................................................................................
15.15. Meta-data in Attributes ....................................................................................
15.16. Namespace for Additional Error Information .......................................................
The CLI agent ...........................................................................................................
16.1. Overview ........................................................................................................
16.2. The J-style CLI ...............................................................................................
16.3. The C- and I-style CLI .....................................................................................
16.4. The CLI in action ............................................................................................
16.5. Environment for OS command execution .............................................................
16.6. Command output processing ..............................................................................
16.7. Range expressions ............................................................................................
16.8. Autorendering of enabled/disabled ......................................................................
16.9. Actions ...........................................................................................................

v

165
166
167
168
170
170
170
170
171
171
171
172
173
173
174
174
176
176
176
177
178
186
187
200
200
200
201
202
202
203
206
208
209
214
214
218
219
222
227
237
239
240
244
245
248
248
249
251
252
254
254
261
263
263

ConfD User Guide

17.

18.

19.

20.

16.10. Command history ........................................................................................... 264
16.11. Clearing history ............................................................................................. 264
16.12. Command line editing ..................................................................................... 264
16.13. Using CLI completion ..................................................................................... 266
16.14. Using the comment characters # or ! .................................................................. 269
16.15. Annotations and tags ....................................................................................... 270
16.16. Activate and Deactivate ................................................................................... 271
16.17. CLI messages ................................................................................................ 272
16.18. confd.conf settings .......................................................................................... 273
16.19. CLI Environment ........................................................................................... 273
16.20. Commands in J-style ....................................................................................... 275
16.21. Commands in C/I-style .................................................................................... 287
16.22. Customizing the CLI ....................................................................................... 299
16.23. User defined wizards ...................................................................................... 305
16.24. User defined wizards in C ................................................................................ 308
16.25. User defined commands in C using the C-API ..................................................... 310
16.26. User defined commands as shell scripts .............................................................. 311
16.27. Modifying built-in commands ........................................................................... 311
16.28. Tailoring show commands ............................................................................... 312
16.29. Change password at initial login ....................................................................... 317
The SNMP Agent ....................................................................................................... 319
17.1. Introduction to the ConfD SNMP Agent .............................................................. 319
17.2. Agent Functional Description ............................................................................. 320
17.3. Generating MIBs from YANG ........................................................................... 339
17.4. Configuring the SNMP Agent ............................................................................ 342
17.5. How the SNMP Agent Interacts with ConfD ......................................................... 346
17.6. Running the SNMP Agent as a NET-SNMP subagent ............................................. 347
Web UI Development ................................................................................................. 349
18.1. Introduction ..................................................................................................... 349
18.2. Example of a common flow ............................................................................... 350
18.3. Example of a JSON-RPC client .......................................................................... 354
18.4. Example of a Comet client ................................................................................ 357
The JSON-RPC API ................................................................................................... 361
19.1. JSON-RPC ...................................................................................................... 361
19.2. Methods - commands ........................................................................................ 366
19.3. Methods - commands - subscribe ........................................................................ 368
19.4. Methods - data ................................................................................................ 373
19.5. Methods - data - attrs ....................................................................................... 375
19.6. Methods - data - leafs ....................................................................................... 376
19.7. Methods - data - leafref .................................................................................... 378
19.8. Methods - data - lists ........................................................................................ 379
19.9. Methods - data - query ...................................................................................... 382
19.10. Methods - database ......................................................................................... 386
19.11. Methods - general ........................................................................................... 387
19.12. Methods - messages ........................................................................................ 390
19.13. Methods - rollbacks ........................................................................................ 391
19.14. Methods - schema .......................................................................................... 393
19.15. Methods - session ........................................................................................... 400
19.16. Methods - session data .................................................................................... 402
19.17. Methods - transaction ...................................................................................... 403
19.18. Methods - transaction - changes ........................................................................ 406
19.19. Methods - transaction - commit changes ............................................................. 408
19.20. Methods - transaction - webui .......................................................................... 410
The web server .......................................................................................................... 411

vi

ConfD User Guide

21.

22.

23.

24.

25.

26.

27.

20.1. Introduction .....................................................................................................
20.2. Web server capabilities .....................................................................................
20.3. CGI support ....................................................................................................
The REST API ..........................................................................................................
21.1. Introduction .....................................................................................................
21.2. Getting started .................................................................................................
21.3. Resource Examples ..........................................................................................
21.4. Resources .......................................................................................................
21.5. Configuration Meta-Data ...................................................................................
21.6. Request/Response headers .................................................................................
21.7. Special characters .............................................................................................
21.8. Error Responses ...............................................................................................
21.9. The Query API ................................................................................................
21.10. Custom Response HTTP Headers ......................................................................
21.11. HTTP Status Codes ........................................................................................
The RESTCONF API ..................................................................................................
22.1. Introduction .....................................................................................................
22.2. Getting started .................................................................................................
22.3. Capabilities .....................................................................................................
22.4. Streams ..........................................................................................................
22.5. Schema resource ..............................................................................................
22.6. YANG Patch Media Type .................................................................................
22.7. Extensions ......................................................................................................
22.8. Custom Response HTTP Headers .......................................................................
The Management Agent API ........................................................................................
23.1. What is MAAPI? .............................................................................................
23.2. A custom toy CLI ............................................................................................
High Availability ........................................................................................................
24.1. Introduction to ConfD High Availability ..............................................................
24.2. HA framework requirements ..............................................................................
24.3. Mode of operation ............................................................................................
24.4. Security aspects ...............................................................................................
24.5. API ................................................................................................................
24.6. Ticks ..............................................................................................................
24.7. Joining a cluster ...............................................................................................
24.8. Relay slaves ....................................................................................................
24.9. CDB replication ...............................................................................................
The SNMP Gateway ...................................................................................................
25.1. Introduction to the ConfD SNMP Gateway ...........................................................
25.2. Configuring Agent Access .................................................................................
25.3. Compiling the MIBs .........................................................................................
25.4. Receiving and Forwarding Notifications ...............................................................
25.5. Example Scenario ............................................................................................
Subagents and Proxies .................................................................................................
26.1. Introduction .....................................................................................................
26.2. Subagent Registration .......................................................................................
26.3. Subagent Requirements .....................................................................................
26.4. Proxies ...........................................................................................................
Plug-and-play scripting ................................................................................................
27.1. Introduction .....................................................................................................
27.2. Script storage ..................................................................................................
27.3. Script interface ................................................................................................
27.4. Loading of scripts ............................................................................................
27.5. Command scripts .............................................................................................

vii

411
411
411
414
414
414
424
438
448
449
450
452
454
459
460
462
462
462
464
464
465
465
465
466
467
467
467
474
474
475
475
477
477
479
479
480
481
482
482
482
483
483
484
485
485
486
490
490
496
496
496
496
497
497

ConfD User Guide

27.6. Policy scripts ................................................................................................... 502
27.7. Post-commit scripts .......................................................................................... 505
28. Advanced Topics ........................................................................................................ 508
28.1. Datastores ....................................................................................................... 508
28.2. Locks ............................................................................................................. 510
28.3. Installing ConfD on a target system .................................................................... 512
28.4. Configuring ConfD ........................................................................................... 513
28.5. Starting ConfD ................................................................................................ 515
28.6. ConfD IPC ...................................................................................................... 517
28.7. Restart strategies .............................................................................................. 521
28.8. Security issues ................................................................................................. 522
28.9. Running ConfD as a non privileged user .............................................................. 523
28.10. Storing encrypted values in ConfD .................................................................... 524
28.11. Disaster management ...................................................................................... 527
28.12. Troubleshooting ............................................................................................. 529
28.13. Tuning the size of confd_hkeypath_t .................................................................. 534
28.14. Error Message Customization ........................................................................... 535
28.15. Using a different version of OpenSSL ................................................................ 535
28.16. Using shared memory for schema information ..................................................... 536
28.17. Running application code inside ConfD .............................................................. 538
I. ConfD man-pages, Volume 1 .......................................................................................... 541
confd ..................................................................................................................... 542
confd_aaa_bridge ..................................................................................................... 547
confd_cli ................................................................................................................ 549
confd_cmd ............................................................................................................. 552
confd_load .............................................................................................................. 554
confdc .................................................................................................................... 559
maapi ..................................................................................................................... 570
II. ConfD man-pages, Volume 3 ......................................................................................... 575
confd_lib ................................................................................................................ 576
confd_lib_cdb .......................................................................................................... 577
confd_lib_dp ........................................................................................................... 615
confd_lib_events ...................................................................................................... 675
confd_lib_ha ........................................................................................................... 682
confd_lib_lib ........................................................................................................... 684
confd_lib_maapi ...................................................................................................... 706
confd_types ............................................................................................................. 773
III. ConfD man-pages, Volume 5 ....................................................................................... 814
clispec .................................................................................................................... 815
confd.conf .............................................................................................................. 872
mib_annotations ....................................................................................................... 930
tailf_yang_cli_extensions ........................................................................................... 932
tailf_yang_extensions ................................................................................................ 973
Glossary ....................................................................................................................... 1009

viii

List of Tables
3.1. YANG built-in types .................................................................................................... 12
17.1. SMI mapping to YANG types .................................................................................... 325
17.2. YANG mapping to SMI types .................................................................................... 326
21.1. REST vs NETCONF operations ................................................................................. 416
21.2. Query Parameters ..................................................................................................... 423
21.3. Resources and their Media Types ............................................................................... 439
21.4. Fields of the /api resource .......................................................................................... 440
21.5. Fields of the /api/ resource ......................................................................... 443
21.6. Built in operations .................................................................................................... 443
21.7. Error code vs HTTP Status ........................................................................................ 454
28.1. ConfD Start Phases .................................................................................................. 516
28.2. ConfD Start Phases, running in foreground ................................................................... 516

ix

List of Examples
5.1. a simple server data model, servers.yang .................................................................. 54
5.2. Pseudo code showing several sessions reusing one connection ............................................. 59
5.3. Pseudo code demonstrating how to avoid re-reading the configuration ................................... 62
5.4. Version 1.0 of the forest module .................................................................................... 63
5.5. Initial forest instance document ..................................................................................... 64
5.6. Version 2.0 of the forest module .................................................................................... 65
5.7. Forest instance document after upgrade ........................................................................... 66
5.8. Enabling the developer log ............................................................................................ 67
5.9. Developer log entries resulting from upgrade ................................................................... 67
5.10. Version 1.5 of the servers.yang module ......................................................................... 69
5.11. Writing to an upgrade transaction using MAAPI ............................................................. 71
5.12. Version 2 of the servers.yang module ............................................................................ 72
5.13. The upgrade() function of server_upgrade.c ...................................................... 73
5.14. A YANG module describing a dhcpd server configuration ................................................ 75
6.1. netstat.yang ................................................................................................................ 82
6.2. ARP table YANG module ............................................................................................ 84
6.3. Populated ARP table .................................................................................................... 85
7.1. A list of server structures ............................................................................................ 104
7.2. The smp.yang module ................................................................................................ 108
7.3. get_next() callback for smp.yang .................................................................................. 110
7.4. create() callback for smp.yang ..................................................................................... 111
7.5. remove() callback for smp.yang ................................................................................... 111
7.6. set_elem() callback for smp.yang .................................................................................. 111
7.7. save() utility function ................................................................................................. 112
7.8. write callbacks using accumulate .................................................................................. 113
7.9. prepare() callback using the accumulated write ops .......................................................... 114
7.10. commit() and abort() ................................................................................................ 115
7.11. Code to restore our array from a file ........................................................................... 116
7.12. checkpoint db callbacks ............................................................................................ 117
10.1. full.yang ................................................................................................................. 135
10.2. small.yang .............................................................................................................. 135
10.3. users.yang ............................................................................................................... 140
12.1. Creating a notification socket ..................................................................................... 160
12.2. reading the audit data ............................................................................................... 160
15.1. Example math rpc .................................................................................................... 210
17.1. Simple YANG module .............................................................................................. 323
17.2. Generating and compiling YANG from MIB ................................................................ 328
17.3. The YANG file generated by confdc --mib2yang .......................................................... 328
17.4. Specifying built-in MIBs to be loaded into the agent ...................................................... 331
17.5. SMI definition of an optional object ............................................................................ 333
17.6. YANG definition of an optional leaf ........................................................................... 334
17.7. simple.mib .............................................................................................................. 334
17.8. simple.yang ............................................................................................................. 335
17.9. simple.yang with secondary index ............................................................................... 335
17.10. TruthValue from the SNMPv2-TC ............................................................................. 336
17.11. A typedef for TruthValue ........................................................................................ 336
17.12. Functions for sending notification from C ................................................................... 337
17.13. SNMP varbind structures from confd_maapi.h ....................................................... 337
17.14. Notification registration ........................................................................................... 338
17.15. Sending a coldStart notification ................................................................................ 338
17.16. Sending a notification with a varbind ......................................................................... 338

x

ConfD User Guide

17.17. Example of a confd.conf .......................................................................................... 343
17.18. Old confd.conf content ............................................................................................ 344
17.19. Updated confd.conf content ...................................................................................... 344
17.20. Example community_init.xml ................................................................................... 345
19.1. Method get_value ..................................................................................................... 376
19.2. Method set_value ..................................................................................................... 377
19.3. Method query .......................................................................................................... 382
19.4. Method start_query ................................................................................................... 384
19.5. Method run_query .................................................................................................... 385
19.6. Method reset_query .................................................................................................. 385
19.7. Method stop_query ................................................................................................... 386
19.8. Method comet ......................................................................................................... 388
19.9. Method get_schema .................................................................................................. 396
19.10. Method get_module_prefix_map ............................................................................... 397
19.11. Method run_action .................................................................................................. 399
19.12. Method login ......................................................................................................... 400
19.13. Method logout ....................................................................................................... 401
19.14. Method get_trans .................................................................................................... 403
19.15. Method new_trans .................................................................................................. 405
19.16. Method get_trans_changes ....................................................................................... 406
21.1. ConfD configuration for REST ................................................................................... 414
21.2. ConfD configuration of the authentication cache TTL ..................................................... 415
21.3. ConfD configuration of Client IP via Proxy .................................................................. 415
21.4. Request URI structure ............................................................................................... 416
21.5. Using curl for accessing ConfD .................................................................................. 417
21.6. Get the "sys/interfaces" resource represented as JSON .................................................... 418
21.7. Create a new "sys/routes/inet/route" resource, with JSON payload ..................................... 419
21.8. Replace the "sys/routes/inet/route" resource contents ...................................................... 420
21.9. Update the "sys/routes/inet/route" resource contents ....................................................... 421
21.10. Delete the "sys/routes/inet/route" resource contents ....................................................... 422
21.11. Get options for the "sys" resource ............................................................................. 422
21.12. Get head for the "sys/interfaces/ex:serial" resource ....................................................... 422
21.13. Shallow get for the "sys" resource ............................................................................. 424
21.14. Deep get for the "sys/interfaces/interface" resource ....................................................... 424
21.15. Limit the response .................................................................................................. 426
21.16. Limit the response with select .................................................................................. 426
21.17. The "sys/ntp/server" list (no defaults) ......................................................................... 427
21.18. The "sys/ntp/server" list with all defaults .................................................................... 428
21.19. Creating a "sys/routes/inet/route" resource .................................................................. 428
21.20. Creating a "sys/interfaces/serial/ppp0/multilink" resource ............................................... 429
21.21. Creating a "route" resource using PUT ....................................................................... 429
21.22. The "route" resource after creation ............................................................................ 430
21.23. Replacing a "route" resource using PUT ..................................................................... 430
21.24. The "route" resource after replace ............................................................................. 431
21.25. Creating a "sys/interfaces/serial/ppp0/multilink" resource ............................................... 431
21.26. Creating a "sys/interfaces/serial/ppp0/authentication" resource ........................................ 432
21.27. The "authentication" resource after replace .................................................................. 432
21.28. Updating a "route" resource using PATCH ................................................................. 433
21.29. The "route" resource after update .............................................................................. 433
21.30. Creating a "sys/interfaces/serial/ppp0/multilink" resource ............................................... 434
21.31. Creating a "sys/interfaces/serial/ppp0/authentication" resource ........................................ 434
21.32. The "authentication" resource after update .................................................................. 434
21.33. The "sys/dns/server" list before insert ........................................................................ 435
21.34. Insert=before in the "sys/dns/server" list ..................................................................... 435

xi

ConfD User Guide

21.35. The "sys/dns/server" list after insert ........................................................................... 436
21.36. An "archive-log" action request example .................................................................... 436
21.37. delete the "sys/interfaces/ex:serial" list ....................................................................... 438
21.38. The "sys/interfaces" resource after delete .................................................................... 438
21.39. delete the "sys/interfaces/ex:serial" list with rollback label and comment ........................... 438
21.40. Namespaces in JSON .............................................................................................. 440
21.41. GET the /api resource ............................................................................................. 442
21.42. GET the /api/running resource .................................................................................. 445
21.43. Action in /api/operational ......................................................................................... 445
21.44. GET rollback files information ................................................................................. 446
21.45. GET rollback file content ........................................................................................ 447
21.46. Find and use the rollback operation resource ............................................................... 448
21.47. XML representation of meta-data .............................................................................. 449
21.48. JSON representation of meta-data ............................................................................. 449
21.49. Example of a XML formatted error message ............................................................... 452
21.50. Example of a JSON formatted error message ............................................................... 452
21.51. ConfD configuration for REST ................................................................................. 459
21.52. ............................................................................................................................ 460
22.1. ConfD configuration for REST ................................................................................... 462
22.2. ConfD configuration for RESTCONF .......................................................................... 462
22.3. ConfD configuration of the authentication cache TTL ..................................................... 463
22.4. ConfD configuration of Client IP via Proxy .................................................................. 463
22.5. ConfD RESTCONF capabilities .................................................................................. 464
22.6. ConfD RESTCONF streams ....................................................................................... 464
22.7. ConfD RESTCONF errors during streaming ................................................................. 465
22.8. Use of collections .................................................................................................... 465
22.9. Use of collections .................................................................................................... 466
22.10. ConfD configuration for RESTCONF ........................................................................ 466
23.1. scli.yang YANG module ........................................................................................... 467
24.1. A data model divided into common and node specific subtrees ......................................... 475
25.1. Example snmpgw configuration fragment in confd.conf .................................................. 482
25.2. C code for registering reception of notifications ............................................................. 483
25.3. Example 1 of translating and compiling a MIB .............................................................. 484
26.1. smtp subagent data ................................................................................................... 487
26.2. imap and pop subagent data ....................................................................................... 487
26.3. Equipment subagent data ........................................................................................... 488
26.4. master agent data ..................................................................................................... 488
26.5. Compile the YANG modules at the master ................................................................... 488
26.6. Master agent's confd.conf .......................................................................................... 489
26.7. Proxy configuration .................................................................................................. 490
26.8. Agent replies with forward capability .......................................................................... 492
26.9. Manager issues forward rpc to board-1 ........................................................................ 492
26.10. Manager issues command ........................................................................................ 492
26.11. close-session .......................................................................................................... 493
26.12. Auto login ............................................................................................................. 493
26.13. Forward rpc with auth data ...................................................................................... 494
152. Reloading all xml files in the cdb directory ................................................................... 557
153. Merging in the contents of conf.cli ......................................................................... 557
154. Print interface config and statistics data in cli format ....................................................... 557
155. Using xslt to format output ......................................................................................... 557
156. Using xmllint to pretty print the xml output ................................................................... 557
157. Saving config and operational data to /tmp/conf.xml ................................................ 557
158. Restoring both config and operational data .................................................................... 557
159. Measure how long it takes to fetch config ..................................................................... 557

xii

ConfD User Guide

160. Output all instances in list /foo/table which has ix larger than 10 ............................... 557
161. confd-light.cli ........................................................................................................... 815
162. The servers YANG model ........................................................................................ 1009

xiii

Chapter 1. About the Documentation
1.1. How to Read This Guide
This document provides a wealth of information about ConfD and how to use it for your particular needs.
It is written to be useful both when read front-to-back and also for readers that need to dive into particular
aspects of the many features of ConfD.
Readers that are new to ConfD will learn a lot about how to think about, and apply, the features of ConfD
by reading the first twelve chapters of this guide. They give an overview of the foundations of ConfD and
how they can be used in various types of environments to meet various types of needs. Having read these
chapters will also be useful as a guide during early design decisions to avoid missing out on useful ConfD
features or applying features in a less than optimal way.
The rest of the document provides information about particular parts of ConfD. Time permitting, it is very
useful to read as a whole, but they may also be read selectively depending on which parts of ConfD you
are planning to use.
This document also consists of manual pages. The manual pages are reference information for the various
tools, libraries and configuration files that are included in the ConfD package. They can also be found in
native manual page format in the ConfD release package.

1.2. Getting Documentation
Updated documentation sets are prepared along with ConfD releases and can always be found in the
customer download area or as part of the various deliverables. All releases contains the following updated
documents:
• The ConfD User Guide is this document and is a separate download
• The CHANGES file describes all new features and corrections in the release and is a separate download
• The HIGHLIGHTS document is released with major releases and describes, with examples, all
substantial new features per release. The HIGHLIGHTS document is a separate download.
• The KNOWN_ISSUES file is part of the release package and documents all known open issues at the
time of release
• All ConfD release packages include a README file that describes how to install, set up and get started
with ConfD. The README file is located in the top directory of a ConfD installation.
• The example collection includes a README file that introduces the reader to the wide selection of
examples and what they contain. The README is contained in the examples deliverable.
All of the documents listed above contain information that is essential to the understanding of how to
extract the most value out of ConfD and we urge all our users to read them.

1.3. Formatting Conventions
We use the following text and syntax conventions throughout the documentation:
Operating system references (e.g. commands, environment variables, filenames and command options)
are rendered in fixed-width font

1

About the Documentation

Programming language constructs (e.g. functions, constants and error codes) are rendered in fixedwidth font
Multi-line code snippets and screen output are rendered like this:
# confdc -c test.cs
# confdc -l -o test.fxs test.xso

We use the following admonitions throughout this document:

Tip
This an example of a tip that is used to describe practical information on how to apply, or think
about a certain aspect of the product

Note
This is an example of a note that is used to highlight a particular piece of information

Warning
This is an example of a warning that points out information that needs particular attention to
avoid problems

1.4. Documentation Feedback
We appreciate documentation feedback, comments and suggestions so that we can continuously improve
the documentation and make it more useful. Use the request tracker system to send us your comments
and make sure you include information about which version and what section of the documentation you
are referring to.

2

Chapter 2. An introduction to ConfD
2.1. An on-device software system for
configuration management
Network devices, such as routers, switches or gateways, need to be configured and monitored. A fair
amount of software is embedded in these devices to facilitate configuration and monitoring. This software
typically includes:
• An SNMP agent for monitoring the device (SNMP is in practice almost never used for configuring
devices, although it is possible to do so).
• Software to drive and render a command line interface (CLI).
• A small web server and content making up a device-specific web site, for a web-based user interface
to the device management system.
In addition, the IETF has developed a standard called NETCONF for automated configuration of network
devices. NETCONF allows devices to expose an XML-based API that the network operator can use to set
and get full and partial configuration data sets.
NETCONF solves several management problems that have been lacking standardized solutions. However,
for an engineering organization with limited resources and a tight time schedule introducing/implementing
NETCONF also poses a problem; a whole new management sub-system needs to be implemented and
integrated with the other already existing management components, while time-to-market requirements
remain unchanged.

2.2. ConfD Architecture
Tail-f's ConfD is a device configuration toolkit meant to be integrated as a management sub-system in
network devices, providing:
• An implementation of the NETCONF protocol
• Automatic rendering of northbound interfaces, including CLI, Web UI and NETCONF
• Clustered/fault-tolerant storage of configuration data
• Master-agent/sub-agent framework for NETCONF, CLI, Web UI and SNMP

3

An introduction to ConfD

ConfD as sub-system on a network device
The following figure illustrates where ConfD would reside on, for example, a chassis-based router:

4

An introduction to ConfD

ConfD on a chassis-based router
ConfD executes as a regular Unix daemon on the target device, acting:
• as a NETCONF agent for the NETCONF protocol
• as a Web server for the Web UI
• as a CLI engine for command-line access
• and as an SNMP agent
It also contains a built-in XML configuration database.
The following figure illustrates the overall architecture. The ConfD architecture is modular, with welldefined interfaces between sub-systems.

5

An introduction to ConfD

ConfD architecture
The NETCONF, SNMP, CLI and Web modules are Management Agents. These communicate with
external managers, and provide the managers with a protocol-specific view of the system. The box labeled
Other Agent is e.g. a GUI application or some other management protocol implementation. These other
Agents use the Management Agent API (MAAPI) to talk to the Management Backplane.
The Management Backplane provides an hierarchical view of the configuration and status/statistics data
through the Management Agent API. This API is a session-oriented read/write API to the hierarchical
data, with transaction-like semantics.
Examples of operations in this interface are 'create-subtree', 'get-instance', 'set-instance'. This interface is
used both when the configuration is stored in the built-in ConfD database, and when it is stored in an
external database.
The Management Backplane authenticates incoming requests through an AAA (Authentication,
Authorization, Accounting) plugin API. An AAA plugin authenticates users and authorizes their requests.
ConfD comes with a built-in AAA plugin, which can be replaced by vendor specific code.
In order to actually read and write the device-native configuration data, the sessions in the Management
Backplane use the Database Plugin API. A database plugin has to provide mapping from the hierarchical
view of the data used in the management protocols, to the native view used by the management database.
The management database can either be the integrated management database - called CDB - or some other
database. CDB is a light-weight fault-tolerant distributed XML database. CDB can be used in single or
multi-node systems in master slave configuration. It handles updates to the database schema automatically.

6

An introduction to ConfD

The Managed Objects in the application use the Managed Object API to read their configuration from the
ConfD management database. There is a also a subscription mechanism, which the Managed Objects can
use to react on configuration changes.
ConfD provides language bindings for the callback oriented plugin interfaces in C and Java. In the figure
above, the Database Plugin API and the AAA Plugin API are available in C and Java The normal function
call oriented APIs are available as C or Java APIs.

7

Chapter 3. The YANG Data Modeling
Language
3.1. The YANG Data Modeling Language
YANG is a data modeling language used to model configuration and state data manipulated by a
NETCONF agent. The YANG modeling language is defined in RFC 6020. YANG as a language will not be
described in its entirety here - rather we refer to the IETF RFC text at http://www.ietf.org/rfc/rfc6020.txt.
Another source of information regarding the YANG language is the wiki based web site http://www.yangcentral.org/. For a tutorial on the data modeling capabilities of YANG, see http://www.yang-central.org/
twiki/bin/view/Main/DhcpTutorial.

3.2. YANG in ConfD
In ConfD, YANG is not only used for NETCONF data. On the contrary, YANG is used to describe the
data model as a whole and used by all northbound interfaces.
A YANG module can be directly transformed into a final schema (.fxs) file that can be loaded into ConfD.
Currently all features of the YANG language except the anyxml statement are supported.

3.3. YANG Introduction
This section is a brief introduction to YANG. The exact details of all language constructs is fully described
in RFC 6020.
The ConfD programmer must know YANG well, since all APIs use various paths that are derived from
the YANG datamodel.

3.3.1. Modules and Submodules
A module contains three types of statements: module-header statements, revision statements, and definition
statements. The module header statements describe the module and give information about the module
itself, the revision statements give information about the history of the module, and the definition
statements are the body of the module where the data model is defined.
A module may be divided into submodules, based on the needs of the module owner. The external view
remains that of a single module, regardless of the presence or size of its submodules.
The include statement allows a module or submodule to reference material in submodules, and the
import statement allows references to material defined in other modules.

3.3.2. Data Modeling Basics
YANG defines four types of nodes for data modeling. In each of the following subsections, the example
shows the YANG syntax as well as a corresponding NETCONF XML representation.

3.3.3. Leaf Nodes
A leaf node contains simple data like an integer or a string. It has exactly one value of a particular type,
and no child nodes.

8

The YANG Data Modeling Language

leaf host-name {
type string;
description "Hostname for this system";
}

With XML value representation for example as:
my.example.com

An interesting variant of leaf nodes are typeless leafs.
leaf enabled {
type empty;
description "Enable the interface";
}

With XML value representation for example as:


3.3.4. Leaf-list Nodes
A leaf-list is a sequence of leaf nodes with exactly one value of a particular type per leaf.
leaf-list domain-search {
type string;
description "List of domain names to search";
}

With XML value representation for example as:
high.example.com
low.example.com
everywhere.example.com

3.3.5. Container Nodes
A container node is used to group related nodes in a subtree. It has only child nodes and no value and
may contain any number of child nodes of any type (including leafs, lists, containers, and leaf-lists).
container system {
container login {
leaf message {
type string;
description
"Message given at start of login session";
}
}
}

With XML value representation for example as:


Good morning, Dave

9

The YANG Data Modeling Language




3.3.6. List Nodes
A list defines a sequence of list entries. Each entry is like a structure or a record instance, and is uniquely
identified by the values of its key leafs. A list can define multiple keys and may contain any number of
child nodes of any type (including leafs, lists, containers etc.).
list user {
key "name";
leaf name {
type string;
}
leaf full-name {
type string;
}
leaf class {
type string;
}
}

With XML value representation for example as:

glocks
Goldie Locks
intruder


snowey
Snow White
free-loader


rzull
Repun Zell
tower


3.3.7. Example Module
These statements are combined to define the module:
// Contents of "acme-system.yang"
module acme-system {
namespace "http://acme.example.com/system";
prefix "acme";
organization "ACME Inc.";
contact "joe@acme.example.com";
description
"The module for entities implementing the ACME system.";
revision 2007-06-09 {

10

The YANG Data Modeling Language

description "Initial revision.";
}
container system {
leaf host-name {
type string;
description "Hostname for this system";
}
leaf-list domain-search {
type string;
description "List of domain names to search";
}
container login {
leaf message {
type string;
description
"Message given at start of login session";
}
list user {
key "name";
leaf name {
type string;
}
leaf full-name {
type string;
}
leaf class {
type string;
}
}
}
}
}

3.3.8. State Data
YANG can model state data, as well as configuration data, based on the config statement. When a node
is tagged with config false, its sub hierarchy is flagged as state data, to be reported using NETCONF's
get operation, not the get-config operation. Parent containers, lists, and key leafs are reported also, giving
the context for the state data.
In this example, two leafs are defined for each interface, a configured speed and an observed speed. The
observed speed is not configuration, so it can be returned with NETCONF get operations, but not with getconfig operations. The observed speed is not configuration data, and cannot be manipulated using editconfig.
list interface {
key "name";
config true;
leaf name {
type string;
}
leaf speed {

11

The YANG Data Modeling Language

type enumeration {
enum 10m;
enum 100m;
enum auto;
}
}
leaf observed-speed {
type uint32;
config false;
}
}

3.3.9. Built-in Types
YANG has a set of built-in types, similar to those of many programming languages, but with some
differences due to special requirements from the management domain. The following table summarizes
the built-in types.

Table 3.1. YANG built-in types
Name

Type

Description

binary

Text

Any binary data

bits

Text/Number

A set of bits or flags

boolean

Text

"true" or "false"

decimal64

Number

64-bit fixed point real number

empty

Empty

A leaf that does not have any value

enumeration

Text/Number

Enumerated
strings
associated numeric values

identityref

Text

A reference to an abstract identity

instance-identifier

Text

References a data tree node

int8

Number

8-bit signed integer

int16

Number

16-bit signed integer

int32

Number

32-bit signed integer

int64

Number

64-bit signed integer

leafref

Text/Number

A reference to a leaf instance

string

Text

Human readable string

uint8

Number

8-bit unsigned integer

uint16

Number

16-bit unsigned integer

uint32

Number

32-bit unsigned integer

uint64

Number

64-bit unsigned integer

union

Text/Number

Choice of member types

with

3.3.10. Derived Types (typedef)
YANG can define derived types from base types using the typedef statement. A base type can be either
a built-in type or a derived type, allowing a hierarchy of derived types. A derived type can be used as the
argument for the type statement.

12

The YANG Data Modeling Language

typedef percent {
type uint16 {
range "0 .. 100";
}
description "Percentage";
}
leaf completed {
type percent;
}

With XML value representation for example as:
20

User defined typedefs are useful when we want to name and reuse a type several times. It is also possible
to restrict leafs inline in the data model as in:
leaf completed {
type uint16 {
range "0 .. 100";
}
description "Percentage";
}

3.3.11. Reusable Node Groups (grouping)
Groups of nodes can be assembled into the equivalent of complex types using the grouping statement.
grouping defines a set of nodes that are instantiated with the uses statement:
grouping target {
leaf address {
type inet:ip-address;
description "Target IP address";
}
leaf port {
type inet:port-number;
description "Target port number";
}
}
container peer {
container destination {
uses target;
}
}

With XML value representation for example as:


192.0.2.1
830



13

The YANG Data Modeling Language

The grouping can be refined as it is used, allowing certain statements to be overridden. In this example,
the description is refined:
container connection {
container source {
uses target {
refine "address" {
description "Source IP address";
}
refine "port" {
description "Source port number";
}
}
}
container destination {
uses target {
refine "address" {
description "Destination IP address";
}
refine "port" {
description "Destination port number";
}
}
}
}

3.3.12. Choices
YANG allows the data model to segregate incompatible nodes into distinct choices using the choice and
case statements. The choice statement contains a set of case statements which define sets of schema
nodes that cannot appear together. Each case may contain multiple nodes, but each node may appear in
only one case under a choice.
When the nodes from one case are created, all nodes from all other cases are implicitly deleted. The device
handles the enforcement of the constraint, preventing incompatibilities from existing in the configuration.
The choice and case nodes appear only in the schema tree, not in the data tree or XML encoding. The
additional levels of hierarchy are not needed beyond the conceptual schema.
container food {
choice snack {
mandatory true;
case sports-arena {
leaf pretzel {
type empty;
}
leaf beer {
type empty;
}
}
case late-night {
leaf chocolate {
type enumeration {
enum dark;
enum milk;
enum first-available;
}

14

The YANG Data Modeling Language

}
}
}
}

With XML value reprentation for example as:

first-available


3.3.13. Extending Data Models (augment)
YANG allows a module to insert additional nodes into data models, including both the current module (and
its submodules) or an external module. This is useful e.g. for vendors to add vendor-specific parameters
to standard data models in an interoperable way.
The augment statement defines the location in the data model hierarchy where new nodes are inserted,
and the when statement defines the conditions when the new nodes are valid.
augment /system/login/user {
when "class != 'wheel'";
leaf uid {
type uint16 {
range "1000 .. 30000";
}
}
}

This example defines a uid node that only is valid when the user's class is not wheel.
If a module augments another model, the XML representation of the data will reflect the prefix of the
augmenting model. For example, if the above augmentation were in a module with prefix other, the
XML would look like:

alicew
Alice N. Wonderland
drop-out
1024


3.3.14. RPC Definitions
YANG allows the definition of NETCONF RPCs. The method names, input parameters and output
parameters are modeled using YANG data definition statements.
rpc activate-software-image {
input {
leaf image-name {
type string;
}
}

15

The YANG Data Modeling Language

output {
leaf status {
type string;
}
}
}



acmefw-2.3




The image acmefw-2.3 is being installed.



3.3.15. Notification Definitions
YANG allows the definition of notifications suitable for NETCONF. YANG data definition statements
are used to model the content of the notification.
notification link-failure {
description "A link failure has been detected";
leaf if-name {
type leafref {
path "/interfaces/interface/name";
}
}
leaf if-admin-status {
type ifAdminStatus;
}
}


2007-09-01T10:00:00Z

so-1/2/3.0
up



3.4. Working With YANG Modules
Assume we have a small trivial YANG file test.yang:
module test {
namespace "http://tail-f.com/test";
prefix "t";

16

The YANG Data Modeling Language

container top {
leaf a {
type int32;
}
leaf b {
type string;
}
}
}

Tip
There is an Emacs mode suitable for YANG file editing in the system distribution. It is called
yang-mode.el
We can use confdc compiler to compile the YANG module.
$ confdc -c test.yang

The above command creates an output file test.fxs that is a compiled schema that can be loaded into
the system. The confdc compiler with all its flags is fully described in confdc (1).
There exists a number of standards based auxiliary YANG modules defining various useful data types.
These modules, as well as their accompanying .fxs files can be found in the ${CONFD_DIR}/src/
confd/yang directory in the distribution.
The modules are:
• ietf-yang-types - defining some basic data types such as counters, dates and times.
• ietf-inet-types - defining several useful types related to IP addresses.
Whenever we wish to use any of those predefined modules we need to not only import the module into our
YANG module, but we must also load the corresponding .fxs file for the imported module into the system.
So if we extend our test module so that it looks like:
module test {
namespace "http://tail-f.com/test";
prefix "t";
import ietf-inet-types {
prefix inet;
}
container top {
leaf a {
type int32;
}
leaf b {
type string;
}
leaf ip {
type inet:ipv4-address;
}
}
}

17

The YANG Data Modeling Language

Normally when importing other YANG modules we must indicate through the --yangpath flag to
confdc where to search for the imported module. In the special case of the standard modules, this is not
required.
We compile the above as:
$ confdc -c test.yang
$ confdc --get-info test.fxs
fxs file
Confdc version:
"3.0_2" (current Confdc version = "3.0_2")
uri:
http://tail-f.com/test
id:
http://tail-f.com/test
prefix:
"t"
flags:
6
type:
cs
mountpoint:
undefined
exported agents:
all
dependencies:
['http://www.w3.org/2001/XMLSchema',
'urn:ietf:params:xml:ns:yang:inet-types']
source:
["test.yang"]

We see that the generated .fxs file has a dependency to the standard
urn:ietf:params:xml:ns:yang:inet-types namespace. Thus if we try to start confd, we
must also ensure that the fxs file for that namespace is loaded.
Failing to do so gives:
$ confd -c confd.conf --foreground --verbose
The namespace urn:ietf:params:xml:ns:yang:inet-types (referenced by http://tail-f.com/test)
Daemon died status=21

The remedy is to modify confd.conf so that it contains the proper load path or to provide the
directory containing the fxs file, alternatively we can provide the path on the command line. The directory
${CONFD_DIR}/etc/confd contains pre-compiled versions of the standard YANG modules.
$ confd -c confd.conf --addloadpath ${CONFD_DIR}/etc/confd --foreground --verbose

confd.conf is the configuration file for ConfD itself. It is described in confd.conf(5)

3.5. Integrity Constraints
The YANG language has built-in declarative constructs for common integrity constraints. These constructs
are conveniently specified as must statements.
A must statement is an XPath expression that must evaluate to true or a non-empty node-set.
An example is:
container interface {
leaf ifType {
type enumeration {
enum ethernet;
enum atm;

18

The YANG Data Modeling Language

}
}
leaf ifMTU {
type uint32;
}
must "ifType != 'ethernet' or "
+ "(ifType = 'ethernet' and ifMTU = 1500)" {
error-message "An ethernet MTU must be 1500";
}
must "ifType != 'atm' or "
+ "(ifType = 'atm' and ifMTU <= 17966 and ifMTU >= 64)" {
error-message "An atm MTU must be 64 .. 17966";
}
}

XPath is a very powerful tool here. It is often possible to express most realistic validation
constraints using XPath expressions. Note that for performance reasons, it is recommended to use the
tailf:dependency statement in the must statement. The compiler gives a warning if a must
statement lacks a tailf:dependency statement, and it cannot derive the dependency from the
expression. The options --fail-on-warnings or -E TAILF_MUST_NEED_DEPENDENCY can be
given to force this warning to be treated as an error. See Section 9.9, “Dependencies - Why Does Validation
Points Get Called” for details.
Another useful built-in constraint checker is the unique statement.
With the YANG code:
list server {
key "name";
unique "ip port";
leaf name {
type string;
}
leaf ip {
type inet:ip-address;
}
leaf port {
type inet:port-number;
}
}

We specify that the combination of IP and port must be unique. Thus the configuration:

smtp
192.0.2.1
25


http
192.0.2.1
25


is not valid.

19

The YANG Data Modeling Language

The usage of leafrefs (See the YANG specification) ensures that we do not end up with configurations
with dangling pointers. Leafrefs are also especially good, since the CLI and Web UI can render a better
interface.
If other constraints are necessary, validation callback functions can be programmed in C. Read more about
validation callbacks in Chapter 9, Semantic validation.

3.6. The when statement
The when statement is used to make its parent statement conditional. If the XPath expression specified as
the argument to this statement evaluates to false, the parent node cannot be given configured. Furthermore,
if the parent node exists, and some other node is changed so that the XPath expression becomes false, the
parent node is automatically deleted. For example:
leaf a {
type boolean;
}
leaf b {
type string;
when "../a = 'true'";
}

This data model snippet says that 'b' can only exist if 'a' is true. If 'a' is true, and 'b' has a value, and 'a' is
set to false, 'b' will automatically be deleted.
Since the XPath expression in theory can refer to any node in the data tree, it has to be re-evaluated when
any node in the tree is modified. But this would have a disastrous performance impact, so in order to avoid
this, ConfD keeps track of dependencies for each when expression. In some simple cases, the confdc can
figure out these dependencies by itself. In the example above, ConfD will detect that 'b' is dependant on 'a',
and evaluate b's XPath expression only if 'a' is modified. If confdc cannot detect the dependencies by itself,
it requires a tailf:dependency statement in the when statement. See Section 9.9, “Dependencies Why Does Validation Points Get Called” for details.

3.7. Using the Tail-f Extensions with YANG
Tail-f has an extensive set of extensions to the YANG language that integrates YANG models in ConfD.
For example when we have config false; data, we may wish to invoke user C code to deliver
the statistics data in runtime. To do this we annotate the YANG model with a Tail-f extension called
tailf:callpoint.
Alternatively we may wish to invoke user code to validate the configuration, this is also controlled through
an extension called tailf:validate.
All these extensions are handled as normal YANG extensions. (YANG is designed to be extended) We
have defined the Tail-f proprietary extensions in a file ${CONFD_DIR}/src/confd/yang/tailfcommon.yang
Continuing with our previous example, adding a callpoint and a validation point we get:
module test {
namespace "http://tail-f.com/test";
prefix "t";

20

The YANG Data Modeling Language

import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
container top {
leaf a {
type int32;
config false;
tailf:callpoint mycp;
}
leaf b {
tailf:validate myvalcp {
tailf:dependency "../a";
}
type string;
}
leaf ip {
type inet:ipv4-address;
}
}
}

The above module contains a callpoint and a validation point. The exact syntax for all Tail-f extensions
are defined in the tailf-common.yang file.
Note the import statement where we import tailf-common.
When we are using YANG specifications in order to generate Java classes for ConfM, these extensions
are ignored. They only make sense on the device side. It is worth mentioning them though, since EMS
developers will certainly get the YANG specifications from the device developers, thus the YANG
specifications may contain extensions
The man page tailf_yang_extensions(5) describes all the Tail-f YANG extensions.

3.7.1. Using a YANG annotation file
Sometimes it is convenient to specify all Tail-f extension statements in-line in the original YANG module.
But in some cases, e.g. when implementing a standard YANG module, it is better to keep the Tail-f
extension statements in a separate annotation file. When the YANG module is compiled to an fxs file, the
compiler is given the original YANG module, and any number of annotation files.
A YANG annotation file is a normal YANG module which imports the module to annotate. Then the
tailf:annotate statement is used to annotate nodes in the original module. For example, the module
test above can be annotated like this:
module test {
namespace "http://tail-f.com/test";
prefix "t";
import ietf-inet-types {
prefix inet;
}
container top {

21

The YANG Data Modeling Language

leaf a {
type int32;
config false;
}
leaf b {
type string;
}
leaf ip {
type inet:ipv4-address;
}
}
}
module test-ann {
namespace "http://tail-f.com/test-ann";
prefix "ta";
import test {
prefix t;
}
import tailf-common {
prefix tailf;
}
tailf:annotate "/t:top/t:a" {
tailf:callpoint mycp;
}
tailf:annotate "/t:top" {
tailf:annotate "t:b" { // recursive annotation
tailf:validate myvalcp {
tailf:dependency "../t:a";
}
}
}
}

In order to compile the module with annotations, use the -a parameter to confdc:
confdc -c -a test-ann.yang test.yang

3.8. Custom Help Texts and Error Messages
Certain parts of a YANG model are used by northbound agents, e.g. CLI and Web UI, to provide the enduser with custom help texts and error messages.

3.8.1. Custom Help Texts
A YANG statement can be annotated with a description statement which is used to describe the
definition for a reader of the module. This text is often too long and too detailed to be useful as help
text in a CLI. For this reason, ConfD by default does not use the text in the description for this
purpose. Instead, a tail-f specific statement, tailf:info is used. It is recommended that the standard
description statement contains a detailed description suitable for a module reader (e.g. NETCONF
client or server implementor), and tailf:info contains a CLI help text.
As an alternative, ConfD can be instructed to use the text in the description statement also for CLI
help text. See the option --use-description to confdc (1).

22

The YANG Data Modeling Language

For example, CLI uses the help text to prompt for a value of this particular type. The CLI shows this
information during tab/command completion or if the end-user explicitly asks for help using the ?character. The behavior depends on the mode the CLI is running in.
The Web UI uses this information likewise to help the end-user.
The mtu definition below has been annotated to enrich the end-user experience:
leaf mtu {
type uint16 {
range "1 .. 1500";
}
description
"MTU is the largest frame size that can be transmitted
over the network. For example, an Ethernet MTU is 1,500
bytes. Messages longer than the MTU must be divided
into smaller frames.";
tailf:info
"largest frame size";
}

3.8.2. Custom Help Text in a Typedef
Alternatively, we could have provided the help text in a typedef statement as in:
typedef mtuType {
type uint16 {
range "1 .. 1500";
}
description
"MTU is the largest frame size that can be transmitted over the
network. For example, an Ethernet MTU is 1,500
bytes. Messages longer than the MTU must be
divided into smaller frames.";
tailf:info
"largest frame size";
}
leaf mtu {
type mtuType;
}

If there is an explicit help text attached to a leaf, it overrides the help text attached to the type.

3.8.3. Custom Error Messages
A statement can have an optional error-message statement. The north-bound agents, for example, the CLI
uses this to inform the end-user about a provided value which is not of the correct type. If no custom errormessage statement is available ConfD generates a built-in error message, e.g. "1505 is too large.".
All northbound agents use the extra information provided by an error-message statement.
The typedef statement below has been annotated to enrich the end-user experience when it comes to
error information:
typedef mtuType {

23

The YANG Data Modeling Language

type uint32 {
range "1..1500" {
error-message
"The MTU must be a positive number not "
+ "larger than 1500";
}
}
}

3.9. Hidden Data
It is sometimes useful to hide nodes from some of the northbound interfaces. The tailf:export
statement, or the --export compile directive can be used to hide an entire module. It is recommended
to use the tailf:export statement. More fine grained control can be attained with the optional
tailf:hidden statement.
The tailf:hidden statement names a hide group. All nodes belonging to the same hide group are
treated the same way as fas as being hidden or invisible. The hide group name full is given a special
meaning. The full hide group is hidden from all northbound interfaces, not just user interfaces.
A related situation is when some nodes should be displayed to the user only when a certain condition is met.
For example, the "ethernet" subtree should be displayed only when the type of an interface is "ethernet".
This is covered in the subsection "Conditional display" below.

3.9.1. Fully Hidden Nodes
This is nodes that may be useful for the MOs, but should be hidden from all northbound interfaces. An
example is the set of physical network interfaces on a device and their types. This is "static" data, i.e. it
can't be changed by configuration, but it can vary between different models of a device that run the same
software, and the device-specific data can be provided via init file or through MAAPI.
This type of data could also be realized via a separate module where tailf:export is used to limit
the visibility, but being able to have some nodes in the data model hidden while others are not allows
for greater flexibility - e.g. lists in the config data can have hidden child nodes, which get instantiated
automatically along with the visible config nodes.

3.9.2. Hiding Nodes from User Interfaces
This is data that is fully visible to programmatic northbound interfaces such as NETCONF, but normally
hidden from user interfaces such as CLI and Web UI. Examples are data used for experimental or endcustomer-specific features, similar to hidden commands in the CLI but for data nodes.
A user interface may give access to this type of data (and even totally hidden data) if the user executes
an unhide command identifying hide group that should be revealed. After this the nodes belonging to
the hide group appear the same as unhidden data, i.e. they're included in tab completion, listed by show
commands etc.
A hide group can only be unhidden if the group is listed in the confd.conf. This means that a hide
group will be completely hidden to the user interfaces unless it has been explicitly allowed to be unhidden
in confd.conf. A password can optionally be required to unhide a group.



24

The YANG Data Modeling Language

debug
secret


3.9.3. Conditional display
Sometimes it is convenient to hide some CLI commands, or Web UI elements, when certain conditions on
the configuration are met. A typical example is a "discriminated union". One leaf is the type of something,
and depending on the value of this leaf, different containers are visible:

typedef interface-type {
type enumeration {
enum ethernet;
enum atm;
enum appletalk;
}
}
leaf if-type {
type interface-type;
}
container ethernet {
...
}
container atm {
...
}
container appletalk {
...
}

In this example, the "ethernet" container should be visible to the user only when the value of "if-type"
is "ethernet".
This can be accomplished by using the tailf:display-when statement. It contains an XPath
expression which specifies when the node should be displayed:

container ethernet {
tailf:display-when "../if-type = 'ethernet'";
...
}
container atm {
tailf:display-when "../if-type = 'atm'";
...
}
container appletalk {
tailf:display-when "../if-type = 'appletalk'";
...
}

With this data model, the CLI behaves like this:

25

The YANG Data Modeling Language

% show
if-type ethernet;
ethernet {
...
# data for ethernet
}
% set 
Possible completions:
ethernet if-type
% set if-type atm
% set atm ...

# create atm data

% delete 
Possible completions:
ethernet if-type

# NOTE: ethernet NOT present

3.10. An Example: Modeling a List of Interfaces
Say for example that we want to model the interface list on a Linux based device. Running the ip link list
command reveals the type of information we have to model
$ /sbin/ip link list
1: eth0: ; mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:12:3f:7d:b0:32 brd ff:ff:ff:ff:ff:ff
2: lo: ; mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: dummy0:  mtu 1500 qdisc noop
link/ether a6:17:b9:86:2c:04 brd ff:ff:ff:ff:ff:ff

and this is how we want to represent the above in XML:




eth0





00:12:3f:7d:b0:32
ff:ff:ff:ff:ff:ff
1500


lo




00:00:00:00:00:00
00:00:00:00:00:00
16436


26

The YANG Data Modeling Language




An interface or a link has data associated with it. It also has a name, an obvious choice to use as the key
- the data item which uniquely identifies an individual interface.
The structure of a YANG model is always a header, followed by type definitions, followed by the actual
structure of the data. A YANG model for the interface list starts with a header:
module links {
namespace "http://example.com/ns/links";
prefix link;
revision 2007-06-09 {
description "Initial revision.";
}
...

A number of datatype definitions may follow the YANG module header. Looking at the output from /sbin/
ip we see that each interface has a number of boolean flags associated with it, e.g. UP, and NOARP.
One way to model a sequence of boolean flags is as a sequence of statements:
leaf UP {
type boolean;
default false;
}
leaf NOARP {
type boolean;
default false;
}

A better way is to model this as:
leaf UP {
type empty;
}
leaf NOARP {
type empty;
}

We could choose to group these leafs together into a grouping. This makes sense if we wish to use the
same set of boolean flags in more than one place. We could thus create a named grouping such as:
grouping LinkFlags {
leaf UP {
type empty;
}
leaf NOARP {
type empty;
}
leaf BROADCAST {
type empty;
}
leaf MULTICAST {

27

The YANG Data Modeling Language

type empty;
}
leaf LOOPBACK {
type empty;
}
leaf NOTRAILERS {
type empty;
}
}

The output from /sbin/ip also contains Ethernet MAC addresses. These are best represented by the macaddress type defined in the ietf-yang-types.yang file. The mac-address type is defined as:
typedef mac-address {
type string {
pattern '[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}';
}
description
"The mac-address type represents an IEEE 802 MAC address.
This type is in the value set and its semantics equivalent to
the MacAddress textual convention of the SMIv2.";
reference
"IEEE 802: IEEE Standard for Local and Metropolitan Area
Networks: Overview and Architecture
RFC 2579: Textual Conventions for SMIv2";
}

This defines a restriction on the string type, restricting values of the defined type "mac-address" to be
strings adhering to the regular expression [0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5} Thus strings
such as a6:17:b9:86:2c:04 will be accepted.
Queue disciplines are associated with each device. They are typically used for bandwidth management.
Another string restriction we could do is to define an enumeration of the different queue disciplines that
can be attached to an interface.
We could write this as:
typedef QueueDisciplineType {
type enumeration {
enum pfifo_fast;
enum noqueue;
enum noop;
enum htp;
}
}

There are a large number of queue disciplines and we only list a few here. The example serves to show
that using enumerations we can restrict the values of the data set in a way that ensures that data entered
always is valid from a syntactical point of view.
Now that we have a number of usable datatypes, we continue with the actual data structure describing a
list of interface entries:
container links {
list link {

28

The YANG Data Modeling Language

key name;
unique addr;
max-elements 1024;
leaf name {
type string;
}
container flags {
uses LinkFlags;
}
leaf addr {
type yang:mac-address;
mandatory true;
}
leaf brd {
type yang:mac-address;
mandatory true;
}
leaf qdisc {
type QueueDisciplineType;
mandatory true;
}
leaf qlen {
type uint32;
mandatory true;
}
leaf mtu {
type uint32;
mandatory true;
}
}
}

The key attribute on the leaf named "name" is important. It indicates that the leaf is the instance key for
the list entry named "link". All the link leafs are guaranteed to have unique values for their name leafs
due to the key declaration.
If one leaf alone does not uniquely identify an object, we can define multiple keys. At least one leaf must
be an instance key - we cannot have lists without a key.
List entries are ordered and indexed according to the value of the key(s).

3.10.1. Modeling Relationships
A very common situation when modeling a device configuration is that we wish to model a relationship
between two objects. This is achieved by means of the leafref statements. A leafref points to a
child of a list entry which either is defined using a key or unique attribute.
The leafref statement can be used to express three flavors of relationships: extensions, specializations
and associations. Below we exemplify this by extending the "link" example from above.
Firstly, assume we want to put/store the queue disciplines from the previous section in a separate container
- not embedded inside the links container.
We then specify a separate container , containing all the queue disciplines which each refers to a specific
link entry. This is written as:
container queueDisciplines {

29

The YANG Data Modeling Language

list queueDiscipline {
key linkName;
max-elements 1024;
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
leaf type {
type QueueDisciplineType;
mandatory true;
}
leaf length {
type uint32;
}
}
}

The linkName statement is both an instance key of the queueDiscipline list, and at the same time
refers to a specific link entry. This way we can extend the amount of configuration data associated with
a specific link entry.
Secondly, assume we want to express a restriction or specialization on Ethernet link entries, e.g. it should
be possible to restrict interface characteristics such as 10Mbps and half duplex.
We then specify a separate container, containing all the specializations which each refers to a specific
link:
container linkLimitations {
list LinkLimitation {
key linkName;
max-elements 1024;
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
container limitations {
leaf only10Mbs { type boolean;}
leaf onlyHalfDuplex { type boolean;}
}
}
}

The linkName leaf is both an instance key to the linkLimitation list, and at the same time refers
to a specific link leaf. This way we can restrict or specialize a specific link.
Thirdly, assume we want to express that one of the link entries should be the default link. In that case
we enforce an association between a non-dynamic defaultLink and a certain link entry:
leaf defaultLink {
type leafref {
path "/config/links/link/name";
}
}

30

The YANG Data Modeling Language

3.10.2. Ensuring Uniqueness
Key leafs are always unique. Sometimes we may wish to impose further restrictions on objects. For
example, we can ensure that all link entries have a unique MAC address. This is achieved through the
use of the unique statement:
container servers {
list server {
key name;
unique "ip port";
unique "index";
max-elements 64;
leaf name {
type string;
}
leaf index {
type uint32;
mandatory true;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
}
}

In this example we have two unique statements. These two groups ensure that each server has a unique
index number as well as a unique ip and port pair.

3.10.3. Default Values
A leaf can have a static or dynamic default value. Static default values are defined with the default
statement in the data model. For example:
leaf mtu {
type int32;
default 1500;
}

and:
leaf UP {
type boolean;
default true;
}

A dynamic default value means that the default value for the leaf is the value of some other leaf in the data
model. This can be used to make the default values configurable by the user. Dynamic default values are
defined using the tailf:default-ref statement. For example, suppose we want to make the MTU
default value configurable:

31

The YANG Data Modeling Language

container links {
leaf mtu {
type uint32;
}
list link {
key name;
leaf name {
type string;
}
leaf mtu {
type uint32;
tailf:default-ref '../../mtu';
}
}
}

Now suppose we have the following data:

1000

eth0
1500


eth1



In the example above, link eth0 has the mtu 1500, and link eth1 has mtu 1000. Since eth1 does not
have a mtu value set, it defaults to the value of ../../mtu, which is 1000 in this case.

Note
Whenever a leaf has a default value it implies that the leaf can be left out from the XML document,
i.e. mandatory = false.
With the default value mechanism an old configuration can be used even after having added new settings.
Another example where default values are used is when a new instance is created. If all leafs within the
instance have default values, these need not be specified in, for example, a NETCONF create operation.

3.10.4. The Final Interface YANG model
Here is the final interface YANG model with all constructs described above:
module links {
namespace "http://example.com/ns/link";
prefix link;
import ietf-yang-types {
prefix yang;
}

grouping LinkFlagsType {

32

The YANG Data Modeling Language

leaf UP {
type empty;
}
leaf NOARP {
type empty;
}
leaf BROADCAST {
type empty;
}
leaf MULTICAST {
type empty;
}
leaf LOOPBACK {
type empty;
}
leaf NOTRAILERS {
type empty;
}
}
typedef QueueDisciplineType {
type enumeration {
enum pfifo_fast;
enum noqueue;
enum noop;
enum htb;
}
}
container config {
container links {
list link {
key name;
unique addr;
max-elements 1024;
leaf name {
type string;
}
container flags {
uses LinkFlagsType;
}
leaf addr {
type yang:mac-address;
mandatory true;
}
leaf brd {
type yang:mac-address;
mandatory true;
}
leaf mtu {
type uint32;
default 1500;
}
}
}
container queueDisciplines {
list queueDiscipline {
key linkName;
max-elements 1024;
leaf linkName {
type leafref {

33

The YANG Data Modeling Language

path "/config/links/link/name";
}
}
leaf type {
type QueueDisciplineType;
mandatory true;
}
leaf length {
type uint32;
}
}
}
container linkLimitations {
list linkLimitation {
key linkName;
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
container limitations {
leaf only10Mbps {
type boolean;
default false;
}
leaf onlyHalfDuplex {
type boolean;
default false;
}
}
}
}
container defaultLink {
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
}
}
}

If the above YANG file is saved on disk, as links.yang, we can compile and link it using the confdc
compiler:
$ confdc -c links.yang

We now have a ready to use schema file named links.fxs on disk. To actually run this example, we
need to copy the compiled links.fxs to a directory where ConfD can find it.

3.11. More on leafrefs
A leafref is a used to model relationships in the data model, as described in Section 3.10.1, “Modeling
Relationships”. In the simplest case, the leafreaf is a single leaf that references a single key in a list:

34

The YANG Data Modeling Language

list host {
key "name";
leaf name {
type string;
}
...
}
leaf host-ref {
type leafref {
path "../host/name";
}
}

But sometimes a list has more than one key, or we need to refer to a list entry within another list. Consider
this example:

list host {
key "name";
leaf name {
type string;
}
list server {
key "ip port";
leaf ip {
type inet:ip-address;
}
leaf port {
type inet:port-number;
}
...
}
}

If we want to refer to a specific server on a host, we must provide three values; the host name, the server
ip and the server port. Using leafrefs, we can accomplish this by using three connected leafs:

leaf server-host {
type leafref {
path "/host/name";
}
}
leaf server-ip {
type leafref {
path "/host[name=current()/../server-host]/server/ip";
}
}
leaf server-port {
type leafref {
path "/host[name=current()/../server-host]"
+ "/server[ip=current()/../server-ip]/../port";
}
}

35

The YANG Data Modeling Language

The path specification for server-ip means the ip address of the server under the host with same name
as specified in server-host.
The path specification for server-port means the port number of the server with the same ip as
specified in server-ip, under the host with same name as specified in server-host.
This syntax quickly gets awkward and error prone. ConfD supports a shorthand syntax, by introducing
an XPath function deref() (see the section called “XPATH FUNCTIONS”). Technically, this function
follows a leafreaf value, and returns all nodes that the leafref refer to (typically just one). The example
above can be written like this:

leaf server-host {
type leafref {
path "/host/name";
}
}
leaf server-ip {
type leafref {
path "deref(../server-host)/../server/ip";
}
}
leaf server-port {
type leafref {
path "deref(../server-ip)/../port";
}
}

Note that using the deref function is syntactic sugar for the basic syntax. The translation between the
two formats is trivial. Also note that deref() is an extension to YANG, and third party tools might not
understand this syntax. In order to make sure that only plain YANG constructs are used in a module, the
parameter --strict-yang can be given to confdc -c.

3.12. Using Multiple Namespaces
There are several reasons for supporting multiple configuration namespaces. Multiple namespaces can be
used to group common datatypes and hierarchies to be used by other YANG models. Separate namespaces
can be used to describe the configuration of unrelated sub-systems, i.e. to achieve strict configuration data
model boundaries between these sub-systems.
As an example, datatypes.yang is a YANG module which defines a reusable data type.
module datatypes {
namespace "http://example.com/ns/dt";
prefix dt;
grouping countersType {
leaf recvBytes {
type uint64;
mandatory true;
}
leaf sentBytes {
type uint64;
mandatory true;
}
}

36

The YANG Data Modeling Language

}

We compile and link datatypes.yang into a final schema file representing the http://
example.com/ns/dt namespace:
$ confdc -c datatypes.yang

To reuse our user defined countersType, we must import the datatypes module.
module test {
namespace "http://tail-f.com/test";
prefix "t";
import datatypes {
prefix dt;
}
container stats {
uses dt:countersType;
}
}

When compiling this new module that refers to another module, we must indicate to confdc where to
search for the imported module:
$ confdc -c test.yang --yangpath /path/to/dt

confdc also searches for referred modules in the colon (:) separated path defined by the environment
variable YANG_MODPATH and . (dot) is implicitly included. Thus we can run pyang on test.yang and
it will search for the datatypes.yang file and import it.

3.13. Module Names, Namespaces and
Revisions
We have three different entities that define our configuration data.
• The module name. A system typically consists of several modules. In the future we also expect to see
standard modules in a manner similar to how we have standard SNMP modules.
It is highly recommended to have the vendor name embedded in the module name, similar to how
vendors have their names in proprietary MIB s today.
• The XML namespace. A module defines a namespace. This is an important part of the module header.
For example we have:
module acme-system {
namespace "http://acme.example.com/system";
.....

The namespace string must uniquely define the namespace. It is very important that once we have settled
on a namespace we never change it. The namespace string should remain the same between revisions of

37

The YANG Data Modeling Language

a product. Do not embed revision information in the namespace string since that breaks manager side
NETCONF scripts.
• The revision statement as in:
module acme-system {
namespace "http://acme.example.com/system";
prefix "acme";
revision 2007-06-09;
.....

The revision is exposed to a NETCONF manager in the capabilities sent from the agent to the NETCONF
manager in the initial hello message. The fine details of revision management is being worked on in the
IETF NETMOD working group and is not finalized at the time of this writing.
What is clear though, is that a manager should base its version decisions on the information in the
revision string.
A capabilities reply from a NETCONF agent to the manager may look as:



urn:ietf:params:netconf:base:1.0
urn:ietf:params:netconf:capability:writable-running:1.0
urn:ietf:params:netconf:capability:candidate:1.0
urn:ietf:params:netconf:capability:confirmed-commit:1.0
urn:ietf:params:netconf:capability:xpath:1.0
urn:ietf:params:netconf:capability:validate:1.0
urn:ietf:params:netconf:capability:rollback-on-error:1.0
http://example.com/ns/link?revision=2007-06-09
....

where the revision information for the http://example.com/ns/link namespace is encoded as
?revision=2007-06-09 using standard URI notation.
When we change the data model for a namespace, it is recommended to change the revision statement,
and to never make any changes to the data model that are backwards incompatible. This means that
all leafs that are added must be either optional or have a default value. That way it is ensured that old
NETCONF client code will continue to function on the new data model. Section 10 of RFC 6020 defines
exactly what changes can be made to a data model in order to not break old NETCONF clients.

3.14. Hash Values and the id-value Statement
Internally and in the programmer APIs, ConfD uses integer values to represent YANG node names and
the namespace URI. This conserves space and allows for more efficient comparisons (including switch
statements) in the user application code. By default, confdc automatically computes a hash value for the
namespace URI and for each string that is used as a node name.
Conflicts can occur in the mapping between strings and integer values - i.e. the initial assignment of integers
to strings is unable to provide a unique, bi-directional mapping. Such conflicts are extremely rare (but
possible) when the default hashing mechanism is used.
The conflicts are detected either by confdc or by the ConfD daemon when it loads the .fxs files.

38

The YANG Data Modeling Language

If there are any conflicts reported they will pertain to XML tags (or the namespace URI),
There are two different cases:
• Two different strings mapped to the same integer. This is the classical hash conflict - extremely rare due
to the high quality of the hash function used. The resolution is to use the tailf:id-value statement
to assign a different integer to one of the strings - we should choose a number above 2^31+1 to avoid
collisions with the generated hash values. To assign a different integer to the namespace URI, we need
to use tailf:id-value as a substatement to the module statement.
• One string mapped to two different integers. This is even more rare than the previous case - it can only
happen if a hash conflict was detected and avoided through the use of tailf:id-value on one of
the strings, and that string also occurs somewhere else. The resolution is to use tailf:id-value to
assign the same integer on both occurrences of the string. As in the previous case, we should choose
a number above 2^31+1.

3.15. Migrating from Confspecs to YANG
There are some constructs in confspecs that are not possible to directly translate into YANG models.
Probably the most problematic is how we, using confspecs, can mount pieces of configuration in one
confspec file into a mount point in another confspec file. It is also possible to mount entire namespaces, .fxs
files, into mount points in other namespaces. Neither is possible with the YANG language.
YANG has the submodule concept as a construct to divide a module. If the overall goal with confspec
mounting is to let different developers work on specific files that are subsequently linked together, that
same goal can be achieved in three different ways with YANG. The method described in Section 3.15.3,
“Partitioning the Module Through Augments Statements” is probably the closest match for the confspec
mounting functionality, but one of the others may be more appropriate or convenient to use.

3.15.1. Submodules and the Uses Statement
The strategy here is have a top module, that simply consists of a series of include statements, a series of
uses statements, and a set of submodules that all define their respective groupings:
module system {
namespace "http://tail-f.com/system";
prefix "s";
include system-sub;
uses top;
uses sys;
}

And then we can have arbitrary many sub modules.
submodule system-sub {
belongs-to system {
prefix s;
}
import ietf-inet-types {
prefix inet;
}

39

The YANG Data Modeling Language

grouping top {
container top {
leaf foo {
type int32;
}
leaf addr{
type inet:ip-address;
}
}
}
grouping sys {
leaf bar {
type int32;
}
}
}

3.15.2. Submodules and toplevel structures
The strategy here is have a top module, that simply consists of a series of include statements, and then
each of the submodules define top level structures.
module system {
namespace "http://tail-f.com/system";
prefix "s";
include system-sub;
include system-sub2;
.....
}

And then we can have arbitrary many sub modules.
submodule system-sub {
belongs-to system {
prefix s;
}
import ietf-inet-types {
prefix inet;
}
container top {
leaf foo {
type int32;
}
leaf addr{
type inet:ip-address;
}
}
leaf bar {
type int32;
}

40

The YANG Data Modeling Language

}

The include statement in the top module will ensure that the structures defined in the submodules(s)
get expanded.

3.15.3. Partitioning the Module Through Augments
Statements
First we need a toplevel file that includes all the submodules:
module system {
namespace "http://tail-f.com/system";
prefix "s";
include system-common;
include system-sub;
}

Following that we have one submodule that defines some shared data types, and most important, the
structure of the entire data model as a skeleton.
submodule system-common {
belongs-to system {
prefix s;
}
import ietf-inet-types {
prefix inet;
}
typedef myType {
type int32;
}
container system {
container chassis {
}
container stats {
}
}
}

And finally, we have one or more submodules that augment the structure defined in systemcommon.yang
submodule system-sub {
belongs-to system {
prefix s;
}

41

The YANG Data Modeling Language

import ietf-inet-types {
prefix inet;
}
include system-common;
augment "/system/chassis" {
leaf foo {
type int32;
}
leaf addr{
type inet:ip-address;
}
}
augment "/s:system/s:stats" {
leaf bar {
type int32;
}
}
}

3.15.4. Miscellaneous
The confspec language has a construct called indexedView. This is a means to indicate in the data
model that items in a list have an order. Some of the north bound agents can then insert new list entries
given by the integer key values. Typical examples are things like firewall rules.
The indexedView concept is supported by the tailf:indexed-view extension to YANG.
However in most cases, it will probably be more appropriate to use the standard, more powerful YANG
mechanism called ordered-by user. Thus confspec users that use the indexedView may want to
migrate to ordered-by user instead of tailf:indexed-view. Read more about ordered-by
user in the YANG specification.
An alternative is to continue to use the Tail-f proprietary indexedView feature in the YANG model.
Yet another area that differs slightly are confspec keyrefs. A confspec keyref uses tagpaths to indicate
what it points to. The YANG equivalent is leafref that behave in a similar way. The major difference
is that a leafref use an XPath expression instead of a tagpath.
Leafrefs that are not also keys may be on the tagpath form we have in confspecs, but they may also be
instantiated XPath expressions - thus making leafrefs more powerful that confspec keyrefs.
We may also have list entries where one or more of the keys are leafrefs. As long as we have list entries
with a single key there is no difference at all compared to confspecs. It is only when we have multiple keys
that we see a difference. See Section 3.11, “More on leafrefs” for more info.

3.15.5. Enumerations
confspecs have enumeration on the form of:






42

The YANG Data Modeling Language



The equivalent enumeration in YANG looks as:
typedef YesNo {
type enumeration {
enum yes;
enum no;
}
}

These two enumerations are however not completely equivalent. YANG enumerations are implicitly
numbered, starting at zero. Thus the exactly equivalent confspec enumerations looks like:







Thus, applications that are currently not using the --allow-enum-conflicts flag to confdc may break. The
C API functions to map from hash value to string doesn't work properly for enumerations any longer when
we use YANG models. When we use YANG models, the flag --allow-enum-conflicts is implicitly used
due to the above mentioned fact that YANG enumerations always start at zero.

3.15.6. The Namespace URI
Prior to YANG we have recommended confspec users to embed version information in the string that
identifies a namespace as in:


using for example the targetNamespace attribute. This is no longer recommended practice. The
recommendation is now to never embed version information in the namespace URI, see Section 3.13,
“Module Names, Namespaces and Revisions”.
Using the new naming scheme one would translate the example above into a YANG module like this:
module interfaces {
namespace "http://example.com/ns/interfaces";
prefix interfaces;
...
}

However if we need to support upgrade from a software release that used confspec to a new release that uses
YANG for the same config we have to be careful. In confspec the identifier of a module is the part of the
namespace before the version number. That is, in http://example.com/ns/interfaces/1.1
the identifier is http://example.com/ns/interfaces (in confspec it can also be explicitly set
using the id attribute on the top-level confspec node). This matters to CDB upgrade, in that CDB
will use the identifier to determine if a namespace is the same or not, when upgrading (so that it can

43

The YANG Data Modeling Language

correctly identify http://example.com/ns/interfaces/2.0 as the new version of http://
example.com/ns/interfaces/1.1). When using YANG ConfD (and CDB) considers the whole
namespace string (since it isn't supposed to change between revisions) as the identifier. To help there is a
YANG extension called tailf:id which takes either an empty string (which means “treat the namespace
just as confspec would, and deduce the id from the namespace string”) or an explicit string (the equivalent
of using id in confspec).
In summary: if we need to support upgrade from a previous release and need to keep the exact namespace
string we had in confspec we can translate the initial confspec fragment above to:
module interfaces {
namespace "http://example.com/ns/interfaces/1.1";
prefix interfaces;
tailf:id "";
...
}

3.15.7. The cs2yang Tool
Finally we must mention the cs2yang tool. It is an XSLT based tool that can be used to translate confspec
files into YANG files. There are some constructs that are not directly translatable, but the output from
cs2yang provides an excellent starting point for migration.
Thus the first step in a migration from confspecs to YANG is to use the cs2yang tool on all confspec files
we wish to translate.
Following that, we must decide on a multi-file mount strategy as described above. The generated files
must then subsequently be hand edited into a suitable form.
In many confspec models, the namespace URI contains a version number. In YANG, the namespace URI
of a data model must not change if the model is updated. Instead, YANG has a revision statement to
handle data model versioning. Unless confspec based systems need to be upgraded in the field to YANG,
it is recommended that the version number in the namespace URI is removed, and a revision statement
is added. In this case, the tailf:id statement that cs2yang generated, must also be removed from the
YANG module. See Section 3.15.6, “The Namespace URI” for a more detailed discussion of this problem.

3.16. The pyang tool
pyang is a Python based tool that is integrated into the confdc tool chain. Pyang can be used to check
and transform YANG models. It has an extensible plugin architecture that allows users to plugin their
own output modules. Thus if we wish to to transform YANG data models - processing e.g proprietary
extensions, writing a Python plugin to pyang is a feasible way forward.
Pyang is developed by Tail-f, and it is open source. Code and documentation are available at http://
www.yang-central.org/.
Pyang is also described in the man page ???

44

Chapter 4. Rendering Agents
4.1. Introduction
In this chapter we reintroduce the links.yang model from the Yang chapter and see how we can use
that model to render all northbound interfaces.
This chapter is an overview, and all concepts touched upon in this chapter are thoroughly described in
later chapters.

4.2. Data Model
The links.yang data model introduced in the "Yang" chapter defines a set of interfaces. Compiling the
links.yang file into an .fxs file is the first required step.
module links {
namespace "http://example.com/ns/link";
prefix link;
import ietf-yang-types {
prefix yang;
}

grouping LinkFlagsType {
leaf UP {
type empty;
}
leaf NOARP {
type empty;
}
leaf BROADCAST {
type empty;
}
leaf MULTICAST {
type empty;
}
leaf LOOPBACK {
type empty;
}
leaf NOTRAILERS {
type empty;
}
}
typedef QueueDisciplineType {
type enumeration {
enum pfifo_fast;
enum noqueue;
enum noop;
enum htb;
}
}
container config {
container links {

45

Rendering Agents

list link {
key name;
unique addr;
max-elements 1024;
leaf name {
type string;
}
container flags {
uses LinkFlagsType;
}
leaf addr {
type yang:mac-address;
mandatory true;
}
leaf brd {
type yang:mac-address;
mandatory true;
}
leaf mtu {
type uint32;
default 1500;
}
}
}
container queueDisciplines {
list queueDiscipline {
key linkName;
max-elements 1024;
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
leaf type {
type QueueDisciplineType;
mandatory true;
}
leaf length {
type uint32;
}
}
}
container linkLimitations {
list linkLimitation {
key linkName;
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
container limitations {
leaf only10Mbps {
type boolean;
default false;
}
leaf onlyHalfDuplex {
type boolean;
default false;
}
}

46

Rendering Agents

}
}
container defaultLink {
leaf linkName {
type leafref {
path "/config/links/link/name";
}
}
}
}
}

The command to compile the YANG file is confdc -c links.yang Thus in our Makefile we have:

all:

links.fxs

%.fxs: %.yang
$(CONFD_DIR)/bin/confdc -c $*.yang

The Makefile requires the UNIX environment variable CONFD_DIR to be set to the directory where ConfD
is installed.
Once we have the .fxs file, it's time to start ConfD. To do that we have some initial steps that must be
taken care of first.
We must have a configuration file for ConfD it self. This file is usually referred to as confd.conf. There
is a multitude of things we can configure in confd.conf(5) but for this initial example we'll just focus on
the things we need to get the links.yang example to work. We can copy the confd.conf file from
etc/confd/confd.conf relative to the installation directory of ConfD and add "." to the loadPath
Now with our newly compiled links.yang file we can start ConfD as:

# source /path/to/installed_confd/confdrc
# confd --foreground --verbose -c ./confd.conf

This starts ConfD in the foreground with all log messages displayed on stdout. This is a convenient way
of running ConfD during development.

4.3. Using the CLIs
The first thing we can try out is how a Juniper style CLI looks and feels on our data model. The ConfD
CLI(s) are entirely rendered from the data model. It's possible to extend and tweak the looks of the CLI
in several ways, but for now, we try just what we get from the plain data model. The ConfD CLI is run
through a small C program called confd_cli which is typically used as a login shell on the target host.
During development, it's usually more convenient to just invoke that program directly from the UNIX
prompt. The confd_cli program communicates with the ConfD daemon over the loopback socket.
Here is an actual session:

# confd_cli

47

Rendering Agents

Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on buzz
admin@buzz 17:37:17> configure
Entering configuration mode private
[ok][2009-03-17 17:37:26]
[edit]
admin@buzz 17:37:26% set config links link eth0 addr 00:12:3f:7d:b0:32 brd 00:1
2:3f:7d:b0:32
[ok][2009-03-17 17:37:48]
[edit]
admin@buzz 17:37:48% set config links link eth0 flags BROADCAST
[ok][2009-03-17 17:38:05]
[edit]
admin@buzz 17:38:05% set config links link eth0 flags
Possible completions:
BROADCAST LOOPBACK MULTICAST NOARP NOTRAILERS UP
admin@buzz 17:38:05% set config links link eth0 flags LOOPBACK
[ok][2009-03-17 17:38:25]
[edit]
admin@buzz 17:38:25% commit
Commit complete.
[ok][2009-03-17 17:38:31]
[edit]
admin@buzz 17:38:31% exit
[ok][2009-03-17 17:38:34]
admin@buzz 17:38:34> show configuration config
links {
link eth0 {
flags {
UP;
BROADCAST;
LOOPBACK;
}
addr 00:12:3f:7d:b0:32;
brd 00:12:3f:7d:b0:32;
}
}
[ok][2009-03-17 17:38:44]
admin@buzz 17:38:44> exit
#

Thus from the data model we got a fully functional Juniper like CLI.
We can also get a Cisco like CLI towards the same data model:
# confd_cli -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on buzz
buzz# show running-config config
config links link eth0
flags UP
flags BROADCAST
flags LOOPBACK
addr 00:12:3f:7d:b0:32

48

Rendering Agents

brd 00:12:3f:7d:b0:32
!
buzz# config
Entering configuration mode terminal
buzz(config)# config links link 
Possible completions:
 eth0
buzz(config)# config links link eth0 
Possible completions:
addr brd flags mtu 
buzz(config)# config links link eth0 mtu 1200
buzz(config-link-eth0)# commit
Commit complete.
buzz(config-link-eth0)#
buzz(config)# config 
Possible completions:
defaultLink linkLimitations links queueDisciplines
buzz(config)# config defaultLink linkName eth0
buzz(config)# commit
Commit complete.

The rendered CLIs are highly capable containing all features expected by a modern CLI.
The data model contains a top level config container. This makes sense from a data modeling perspective
since it wraps all configuration items inside a container. However, we may wish to do away with the
container in the CLI. We want to issue the command defaultLink linkName eth0 as opposed to
the command config defaultLink linkName eth0.
The ConfD CLI can be tweaked in a myriad different ways. The typical development cycle is to define
the data model to be as succinct and understandable as possible. This makes life easier for the application
programmers who write C/C++ code that access the data model. Once the YANG model is good, we tweak
the CLI to become what we want.
In this tiny example we write a really small CLI modification file:











Save that file as mods.cli and compile that file as:

# confdc -c mods.cli

This results in a file called mods.ccl that needs to be put in the load path of ConfD. We then re-launch
the Cisco CLI:

49

Rendering Agents

# confd_cli -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on buzz
buzz# config
Entering configuration mode terminal
buzz(config)# links link eth0 
Possible completions:
addr brd flags mtu 
buzz(config)# links link eth0 mtu ?
Possible completions:
[1200]
buzz(config)# links link eth0 mtu 1400
buzz(config-link-eth0)# commit
Commit complete.
buzz(config-link-eth0)#

4.4. Using NETCONF
NETCONF is a powerful protocol that can be used to programmatically reconfigure a device. We're still
running ConfD with the links.yang data model.
The easiest way to interact with the NETCONF agent in ConfD is to use a small python based program
called netconf-console that ships with ConfD. Let's just run it from the UNIX prompt and see what we get:

# netconf-console --user=admin --password=admin --get-config -x '/config'






eth0





00:12:3f:7d:b0:32
00:12:3f:7d:b0:32
1400





eth0

true




eth0




50

Rendering Agents



The above command uses the python paramiko ssh client to establish an SSH session to ConfD. It then
issues a NETCONF get-config RPC to retrieve all configuration data found below the XPath /config,
i.e. this is precisely the data we have just entered in the CLI. The command netconf-console --get-config
is a great way to extract a backup of the entire configuration of the device.
If we want to manipulate the configuration with the netconf-consoleprogram , we must prepare some
XML data that can be sent as an edit-config and feed it to netconf-console.
Here is an example, save the following to a file, set-mtu.xml.









eth0
1100






Then run this with:
# netconf-console --user=admin --password=admin --rpc set-mtu.xml





It's also fairly instructive to directly connect to the agent using OpenSSH as in:
# ssh -s -p 2022 admin@localhost netconf
admin@localhost's password:



urn:ietf:params:netconf:base:1.0
urn:ietf:params:netconf:capability:writable-running:1.0
urn:ietf:params:netconf:capability:candidate:1.0
urn:ietf:params:netconf:capability:confirmed-commit:1.0
urn:ietf:params:netconf:capability:xpath:1.0
urn:ietf:params:netconf:capability:url:1.0?scheme=ftp,file
urn:ietf:params:netconf:capability:validate:1.0
urn:ietf:params:netconf:capability:rollback-on-error:1.0
http://example.com/ns/link
http://tail-f.com/ns/aaa/1.1

14

The agent replies with the capabilities it supports.

51

Rendering Agents

4.4.1. Generating Java classes for JNC
It is of course entirely possible to use the netconf-console program to XML script towards a NETCONF
agent. An alternative is to use a Java library such as JNC to interact with the NETCONF agent. JNC is in
the Open SOurce and can by found on github at https://github.com/tail-f-systems/JNC/.

52

Chapter 5. CDB - The ConfD XML
Database
5.1. Introduction
This chapter describes how to use ConfD's built-in configuration database CDB. As a running example,
we will describe a DHCP daemon configuration. CDB can also be used to store operational data - read
more about this in Section 6.8, “Operational data in CDB”.
A network device needs to store its configuration somewhere. Usually the device configuration is stored
in a database or in plain files, sometimes a combination of both.
ConfD has a built-in XML database which can be used to store the configuration data for the device. The
database is called CDB - Configuration DataBase.

5.2. CDB
By default, ConfD stores all configuration data in CDB. The alternative is to use an external database as
described in Chapter 7, The external database API. There are a number of advantages to CDB compared
to using some external storage for configuration data. CDB has:
• A solid model on how to handle configuration data in network devices, including a good update
subscription mechanism.
• A networked API whereby it is possible for an unconfigured device to find the configuration data on
the network and use that configuration.
• Fast lightweight database access. CDB by default keeps the entire configuration in RAM as well as
on disk.
• Ease of use. CDB is already integrated into ConfD, the database is lightweight and has no maintenance
needs. Writing instrumentation functions to access data is easy.
• Automatic support for upgrade and downgrade of configuration data. This is a key feature, which is
useful not only when performing actual up/downgrades on the device. It also greatly simplifies the
development process by allowing individual developers to add/delete items in the configuration without
any impact whatsoever on other developers. This will be fully described later.
When using CDB to store the configuration data, the applications need to be able to:
1. Read configuration data from the database.
2. React when the database is written to. There are several possible writers to the database, such as the
CLI, NETCONF sessions, the Web UI, or the NETCONF agent. Suppose an operator runs the CLI and
changes the value of some leaf. When this happens, the application needs to be informed about the
configuration change.
The following figure illustrates the architecture when CDB is used.

53

CDB - The ConfD XML Database

ConfD CDB architecture scenario
The Applications/Managed Objects in the figure above read configuration data and subscribe to changes to
the database using a simple RPC-based API. The API is part of the libconfd.so shared library and is
fully documented in the UNIX man page confd_lib_cdb(3). Since the API is RPC-based, the Applications
may run on other hosts that are not running ConfD - which could be used for example in a chassis-based
system where ConfD only would run on the management blade, and the managed applications on other
blades in the system.

5.3. An example
Let us look at a simple example which will illustrate how to populate the database, how to read from it
using the C API, as well as react to changes to the data. First we need a YANG module (see Chapter 3, The
YANG Data Modeling Language for more details about how to write data models in YANG). Consider
this simplified, but functional, example:

Example 5.1. a simple server data model, servers.yang
module servers {
namespace "http://example.com/ns/servers";
prefix servers;
import ietf-inet-types {
prefix inet;
}
revision "2006-09-01" {

54

CDB - The ConfD XML Database

description "Initial servers data model";
}
/* A set of server structures
container servers {
list server {
key name;
max-elements 64;
leaf name {
type string;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
}
}

*/

}

Since we are using CDB here, ConfD will keep an XML tree conforming to the above data model in its
internal persistent XML database.
We start by saving the YANG module to a file, servers.yang and compile and link the data model
into a single servers.fxs which is the binary format, used by ConfD, of a YANG module.
$ confdc -c servers.yang

We then proceed to use the --emit-h flag to generate a .h file which contains the namespace symbol
(servers__ns) which we need in order to use the CDB API.
$ confdc --emit-h servers.h servers.fxs
$ head servers.h
#ifndef _SERVERS_H_
#define _SERVERS_H_
#ifndef
#define
#define
#define
#endif

servers__ns
servers__ns 686487091
servers__ns_id "http://example.com/ns/servers"
servers__ns_uri "http://example.com/ns/servers"

Once we have compiled the YANG module, we can start ConfD. We need to provide a configuration file
to ConfD which indicates that we want ConfD to store the configuration.
The relevant parts from the confd.conf configuration file are:

/etc/confd

...

true
/var/confd/cdb


55

CDB - The ConfD XML Database

The newly generated .fxs file must be copied to the directory /etc/confd and the directory /var/
confd/cdb must exist and be writable. Thus:
$ cp servers.fxs /etc/confd
$ mkdir /var/confd/cdb
$ confd -v --foreground

By far the easiest way to populate the database with some actual data is to run the CLI.
$ confd_cli -u admin
admin connected from 127.0.0.1 using console on buzz
admin@buzz> configure private
Entering configuration mode "private"
admin@buzz% set servers server www
admin@buzz% set servers server www port 80
admin@buzz% set servers server www ip 192.168.128.1
admin@buzz% commit
Configuration committed
admin@buzz> show configuration
servers {
server www {
ip 192.168.128.1;
port 80;
}
}

Now the database is populated with a single server instance.
What remains to conclude our simple example is to write our application - our managed object - the code
that uses the configuration data in the database. The implied meaning of the servers.yang YANG
module is that the managed object would start and stop the services in the configuration. We will not do
that; we will merely show how to read the configuration from the CDB database and react to changes in
CDB.
The code is straightforward. We are using the API functions from libconfd.so. The CDB API is fully
described in the UNIX man page confd_lib_cdb(3).
Main looks like this:
int main(int argc, char **argv)
{
struct sockaddr_in addr;
int subsock;
int status;
int spoint;
addr.sin_addr.s_addr = inet_addr("127.0.0.1");
addr.sin_family = AF_INET;
addr.sin_port = htons(CONFD_PORT);
confd_init(argv[0], stderr, CONFD_SILENT);
if ((subsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open socket\n");
if (cdb_connect(subsock, CDB_SUBSCRIPTION_SOCKET,
(struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");
if ((status =

56

CDB - The ConfD XML Database

cdb_subscribe(subsock, 3, servers__ns, &spoint,"/servers"))
!= CONFD_OK) {
fprintf(stderr, "Terminate: subscribe %d\n", status);
exit(1);
}
if (cdb_subscribe_done(subsock) != CONFD_OK)
confd_fatal("cdb_subscribe_done() failed");
if ((status = read_conf(&addr)) != CONFD_OK) {
fprintf(stderr, "Terminate: read_conf %d\n", status);
exit(1);
}
/* ... */

The code initializes the library, reads the configuration and creates a socket to CDB. One socket
is a read socket and it is used to read configuration data by means of CDB API read functions,
while the other is a subscription socket. The subscription socket must be part of the client poll()
set. Whenever data arrives on the subscription socket, the client invokes a CDB API function,
cdb_read_subscription_socket() on the subscription socket. The subscription model will be
explained further later in this chapter.
The read_conf() function reads the configuration data from CDB and stores it in local ephemeral
(temporary) data structures. We have:
#include "servers.h"
struct server {
char name[BUFSIZ];
struct in_addr ip;
unsigned int port;
};
static struct server running_db[64];
static int num_servers = 0;
static int read_conf(struct sockaddr_in *addr)
{
int rsock, i, n, st = CONFD_OK;
struct in_addr ip;
u_int16_t port;
char buf[BUFSIZ];
if ((rsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
return CONFD_ERR;
if (cdb_connect(rsock, CDB_READ_SOCKET, (struct sockaddr*)addr,
sizeof (struct sockaddr_in)) < 0)
return CONFD_ERR;
if (cdb_start_session(rsock, CDB_RUNNING) != CONFD_OK)
return CONFD_ERR;
cdb_set_namespace(rsock, servers__ns);
num_servers = 0;
if ((n = cdb_num_instances(rsock, "/servers/server")) < 0) {
cdb_close(rsock);
return n;
}
num_servers = n;
for(i=0; i 0) {
if ((status = read_conf(&addr)) != CONFD_OK)
exit(1);
}
print_servers();
/* do something with data here... */
if ((status =
cdb_sync_subscription_socket(subsock, CDB_DONE_PRIORITY))
!= CONFD_OK) {
exit(status);
}

Instead of actually using the data we will merely print it to stdout when we receive any changes:
static void print_servers()
{
int i;
for (i=0; i < num_servers; i++) {
printf("server %d: %s %s:%d\n", i, running_db[i].name,
inet_ntoa(running_db[i].ip), running_db[i].port);
}
}

5.4. Using keypaths
We'll go through all the CDB API functions used in the C code, but first a note on the path notation. Several
of the API functions take a keypath as a parameter. A keypath leads down into the configuration data tree.

58

CDB - The ConfD XML Database

A keypath can be either absolute or relative. An absolute keypath starts from the root of the tree, while a
relative path starts from the "current position" in the tree. They are differentiated by presence or absence
of a leading "/". It is possible to change the "current position" with for example the cdb_cd() function.
XML elements that are containers for other XML elements, such as the servers container that contains
multiple server instances, can be traversed using two different path notations. In our code above, we
use the function cdb_num_instances() to figure out how many children a list has, and then traverse
all children using a [%d] notation. The children of a list have an implicit numbering starting at 0. Thus the
path: /servers/server[2]/port refers to the "port" leaf of the third server in the configuration.
This numbering is only valid during the current CDB session. CDB is always locked for the duration of
the read session.
We can also refer to list instances using the values of the keys of the list. Remember that we specified in the
data model which leaf(s) in the XML structure were keys using the key name statement at the beginning
of the list. In our case a server has the name leaf as key. So the path: /servers/server{www}/
ip refers to the ip leaf of the server whose name is "www".
A YANG list may have more than one key. In the next section we will provide an example where we
configure a DHCP daemon. That data model uses multiple keys and for example the path: /dhcp/
subNets/subNet{192.168.128.0 255.255.255.0}/routers refers to the routers list of
the subNet which has key "192.168.128.0 255.255.255.0".
The syntax for keys is a space separated list of key values enclosed within curly brackets: { Key1
Key2 ...}
Which version of bracket notation to use depends on the situation. For example the bracket notation is
normally used when looping through all instances. As a convenience all functions expecting keypaths
accept formatting characters and accompanying data items. For example cdb_get("server[%d]/
ifc{%s}/mtu", 2, "eth0") to fetch the MTU of the third server instance's interface named "eth0".
Using relative paths and cdb_pushd() it is possible to write code that can be re-used for common subtrees. An example of this is presented further down.
The current position also includes the namespace. To read elements from a different namespace use the
cdb_set_namespace() function.

5.5. A session
It is important to consider that CDB is locked for writing during a read session using the C API. A session
starts with cdb_start_session() and the lock is not released until the cdb_end_session() (or
the cdb_close()) call. CDB will also automatically release the lock if the socket is closed for some
other reason, such as program termination.
In the example above we created a new socket each time we called read_conf(). It is also possible
to re-use an existing connection.

Example 5.2. Pseudo code showing several sessions reusing one connection
cdb_connect(s);
..
cdb_start_session(s); /* Start session and take CDB lock */
cdb_cd();
cdb_get();
cdb_end_session(s);
/* lock is released */

59

CDB - The ConfD XML Database

..
cdb_start_session(s); /* Start session and take CDB lock */
cdb_get();
cdb_end_session(s);
/* lock is released */
..
cdb_close(s);

5.6. CDB subscriptions
The CDB subscription mechanism allows an external program to be notified when different parts of the
configuration changes. At the time of notification it is also possible to iterate through the changes written
to CDB. Subscriptions are always towards the running datastore (it is not possible to subscribe to changes
to the startup datastore). Subscriptions towards the operational data kept in CDB are also possible, but the
mechanism is slightly different, see below.
The first thing to do is to inform CDB which paths we want to subscribe to, registering a path returns a
subscription point identifier. This is done with the cdb_subscribe() function. Each subscriber can
have multiple subscription points, and there can be many different subscribers. Every point is defined
through a path - similar to the paths we use for read operations, with the exception that instead of fully
instantiated paths to list instances we can selectively use tagpaths.
We can subscribe either to specific leaves, or entire subtrees. Explaining this by example we get:
/named/options/pid-file
a subscription to a leaf. Only changes to this leaf will generate a notification.
/servers
Means that we subscribe to any changes in the subtree rooted at /servers. This includes additions
or removals of server instances, as well as changes to already existing server instances.
/servers/server{www}/ip
Means that we only want to be notified when the server "www" changes its ip address.
/servers/server/ip
Means we want to be notified when the leaf ip is changed in any server instance.
When adding a subscription point the client must also provide a priority, which is an integer. As CDB is
changed, the change is part of a transaction. For example the transaction is initiated by a commit operation
from the CLI or a candidate-commit operation in NETCONF resulting in the running database being
modified. As the last part of the transaction CDB will generate notifications in lock-step priority order. First
all subscribers at the lowest numbered priority are handled, once they all have replied and synchronized by
calling cdb_sync_subscription_socket() the next set - at the next priority level - is handled by
CDB. Not until all subscription points have been acknowledged is the transaction complete. This implies
that if the initiator of the transaction was for example a commit command in the CLI, the command will
hang until notifications have been acknowledged.
Note that even though the notifications are delivered within the transaction it is not possible for a subscriber
to reject the changes (since this would break the two-phase commit protocol used by the ConfD backplane
towards all data-providers).
When a client is done subscribing it needs to inform ConfD it is ready to receive notifications. This is done
by first calling cdb_subscribe_done(), after which the subscription socket is ready to be polled.

60

CDB - The ConfD XML Database

As a subscriber has read its subscription notifications using cdb_read_subscription_socket()
it can iterate through the changes that caused the particular subscription notification using
the cdb_diff_iterate() function. It is also possible to start a new read-session to the
CDB_PRE_COMMIT_RUNNING database to read the running database as it was before the pending
transaction.
Subscriptions towards the operational data in CDB are similar to the above, but due to the fact that the
operational data store is designed for light-weight access, and thus does not have transactions and normally
avoids the use of any locks, there are several differences - in particular:
• Subscription notifications are only generated if the writer obtains a "subscription lock", by
using the cdb_start_session2() function with the CDB_LOCK_REQUEST flag, see the
confd_lib_cdb(3) manual page. It is possible to obtain a "subscription lock" for a subtree of the
operational data store by using the CDB_LOCK_PARTIAL flag.
• Subscriptions are registered by using the cdb_subscribe2() function with type
CDB_SUB_OPERATIONAL (or cdb_oper_subscribe()) rather than cdb_subscribe().
• No priorities are used.
• Neither the writer that generated the subscription notifications nor other writes to the same data are
blocked while notifications are being delivered. However the subscription lock remains in effect until
notification delivery is complete.
• The previous value for a modified leaf is not available when using the cdb_diff_iterate()
function.
Essentially a write operation towards the operational data store, combined with the subscription lock,
takes on the role of a transaction for configuration data as far as subscription notifications are concerned.
This means that if operational data updates are done with many single-element write operations, this can
potentially result in a lot of subscription notifications. Thus it is a good idea to use the multi-element
cdb_set_object() etc functions for updating operational data that applications subscribe to.
Since write operations that do not attempt to obtain a subscription lock are allowed to proceed even during
notification delivery, it is the responsibility of the applications using the operational data store to obtain
the lock as needed when writing. E.g. if subscribers should be able to reliably read the exact data that
resulted from the write that triggered their subscription, a subscription lock must always be obtained when
writing that particular set of data elements. One possibility is of course to obtain a lock for all writes to
operational data, but this may have an unacceptable performance impact.
To view registered subscribers use the confd --status command. For details on how to use the different
subscription functions see the confd_lib_cdb(3) manual page.

5.7. Reconnect
If ConfD is restarted, our CDB sockets obviously die. The correct thing to do then is to re-open the cdb
sockets and re-read the configuration. In the case of a high availability setup this also applies. If we are
connected to one ConfD node and that node dies, we must reconnect to another ConfD node and read/
subscribe to the configuration from that node.
If the configuration has not changed we do not want to restart our managed objects, we just want to
reconnect our CDB sockets. The API function cdb_get_txid() will read the last transaction id from
our cdb socket. The id is guaranteed to be unique. We issue the call cdb_get_txid() on the data socket
and we must not have an active read session on that socket while issuing the call.

61

CDB - The ConfD XML Database

Example 5.3. Pseudo code demonstrating how to avoid re-reading the configuration
struct cdb_txid prev_stamp;
cdb_connect(s);
load_config(s);
cdb_get_txid(s, &prev_stamp);
...
subscribe(....);
while (1) {
poll(...);
if (has_new_data(s)) {
load_config(s);
cdb_get_txid(s, &prev_stamp);
}
else if (is_closed(s)) {
struct cdb_txid new_stamp;
cdb_connect(s);
cdb_get_txid(s, &new_stamp);
if ((prev_stamp.s1 == new_stamp.s1) &&
(prev_stamp.s2 == new_stamp.s2) &&
(prev_stamp.s3 == new_stamp.s3) &&
(strcmp(prev_stamp.master, new_stamp.master) == 0)) {
/* no need to re-read config, it hasn't changed */
continue;
}
else {
load_config(s);
cdb_get_txid(s, &prev_stamp);
}
}
}

5.8. Loading initial data into CDB
When ConfD starts for the first time, assuming CDB is enabled, the CDB database is empty. CDB is
configured to store its data in a directory as in:

true
/var/confd/cdb


At startup, when CDB is empty, i.e. no database files are found in the CDB directory, CDB will try
to initialize the database from all instantiated XML documents found in the CDB directory. This is the
mechanism we use to have an empty database initialized to some default setup.
This feature can be used to for example reset the configuration back to some factory setting or some such.
For example, assume we have the data model from Example 5.1, “a simple server data model,
servers.yang”. Furthermore, assume CDB is empty, i.e. no database files at all reside under /var/
confd/cdb. However we do have a file, /var/confd/cdb/foobar.xml containing the following
data:


www
192.168.3.4
88

62

CDB - The ConfD XML Database



www2
192.168.3.5
80


smtp
192.168.3.4
25


dns
192.168.3.5
53



CDB will be initialized from the above XML document. The feature of initializing CDB with some
predefined set of XML elements is used to initialize the AAA database. This is described in Chapter 14,
The AAA infrastructure.
All files ending in .xml will be loaded (in an undefined order) and committed in a single transaction when
CDB enters start phase 1 (see Section 28.5, “Starting ConfD” for more details on start phases). The format
of the init files is rather lax in that it is not required that a complete instance document following the datamodel is present, much like the NETCONF edit-config operation. It is also possible to wrap multiple
top-level tags in the file with a surrounding config tag, like this:

...


5.9. Automatic schema upgrades and
downgrades
Software upgrades and downgrades represent one of the main problems of managing configuration data of
network devices. Each software release for a network device is typically associated with a certain version
of configuration data layout, i.e. a schema. In ConfD the schema is the data model stored in the .fxs files.
Once CDB has initialized it also stores a copy of the schema associated with the data it holds.
Every time ConfD starts, CDB will check the current contents of the .fxs files with its own copy of the
schema files. If CDB detects any changes in the schema it initiates an upgrade transaction. In the simplest
case CDB automatically resolves the changes and commits the new data before ConfD reaches start-phase
one.
An example of how the CDB automatically handles upgrades follows. For version 1.0 of the "forest
configurator" software project the following YANG module is used:

Example 5.4. Version 1.0 of the forest module
module forest {
namespace "http://example.com/ns/forest";
prefix forest;
revision "2006-09-01" {
description "Initial forest model";

63

CDB - The ConfD XML Database

}
container forest {
list tree {
key name;
min-elements 2;
max-elements 1024;
leaf name {
type string;
}
leaf height {
type uint8;
mandatory true;
}
leaf type {
type string;
mandatory true;
}
}
list flower {
key name;
max-elements 1024;
leaf name {
type string;
}
leaf type {
type string;
mandatory true;
}
leaf color {
type string;
mandatory true;
}
}
}
}

The YANG module will be mounted at / in the larger compounded data model tree. We start ConfD and
populate the CDB, e.g. by using the ConfD CLI. The programmer then writes C code which reads these
items from the configuration database.
To demonstrate the automatic upgrade, we will assume that CDB is populated with the instance data in
Example 5.5, “Initial forest instance document”

Example 5.5. Initial forest instance document


George10oak


Eliza15oak


Henry12pine


Sebastiandandelionyellow


64

CDB - The ConfD XML Database


Alvintulipwhite



During the development of the next version of the "forest configurator" software project a couple of
changes were made to the configuration data schema. The tree list height was found to need more than
256 possible values and expanded to a 32-bit integer, and two new leaves color and birthday were
added. The list flower had an optional edible leaf added, and the color changed type to a more strict
enumeration type. The result is in Example 5.6, “Version 2.0 of the forest module”.

Example 5.6. Version 2.0 of the forest module
module forest {
namespace "http://example.com/ns/forest";
prefix forest;
import tailf-xsd-types {
prefix xs;
}
revision "2009-10-01" {
description
"Needed room for taller trees.
Flowers can be edible.";
}
revision "2008-09-01" {
description "Initial forest model";
}
typedef colorType {
type enumeration {
enum unknown;
enum blue;
enum yellow;
enum red;
enum green;
}
}
container forest {
list tree {
key name;
min-elements 2;
max-elements 1024;
leaf name {
type string;
}
leaf height {
type int32;
mandatory true;
}
leaf birthday {
type xs:date;
default 2006-09-01;
}
leaf color {
type colorType;
default unknown;

65

CDB - The ConfD XML Database

}
leaf type {
type string;
mandatory true;
}
}
list flower {
key name;
max-elements 1024;
leaf name {
type string;
}
leaf type {
type string;
mandatory true;
}
leaf edible {
type empty;
}
leaf color {
type colorType;
default unknown;
}
}
}
}

After compiling this new version of the YANG module into an fxs file using confdc and restarting ConfD
with the new schema, CDB automatically detects that the namespace http://example.com/ns/
forest has been modified. CDB will then update the schema and the contents of the database. When
ConfD has started the data in the database now looks like in Example 5.7, “Forest instance document after
upgrade”

Example 5.7. Forest instance document after upgrade


Eliza
15
2006-09-01
unknown
oak


George
10
2006-09-01
unknown
oak


Henry
12
2006-09-01
unknown
pine


Alvin

66

CDB - The ConfD XML Database

tulip
unknown


Sebastian
dandelion
yellow



Let's follow what CDB does by checking the devel log. The devel log is meant to be used as support
while the application is developed. It is enabled in confd.conf as shown in Example 5.8, “Enabling
the developer log”.

Example 5.8. Enabling the developer log

true

true
/var/confd/log/devel.log


trace

Example 5.9. Developer log entries resulting from upgrade
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:
upgrade:

http://example.com/ns/forest -> http://example.com/ns/forest
/forest/flower/{"Alvin"}/color: -> unknown (default because old value white does no
/forest/flower/{"Sebastian"}/color: -> yellow (but with new type)
/forest/tree/{"Eliza"}/birthday: added, with default (2006-09-01)
/forest/tree/{"Eliza"}/color: added, with default (unknown)
/forest/tree/{"Eliza"}/height: -> 15 (but with new type)
/forest/tree/{"George"}/birthday: added, with default (2006-09-01)
/forest/tree/{"George"}/color: added, with default (unknown)
/forest/tree/{"George"}/height: -> 10 (but with new type)
/forest/tree/{"Henry"}/birthday: added, with default (2006-09-01)
/forest/tree/{"Henry"}/color: added, with default (unknown)
/forest/tree/{"Henry"}/height: -> 12 (but with new type)

CDB can automatically handle the following changes to the schema:
Deleted elements
When an element is deleted from the schema, CDB simply deletes it (and any children) from the
database.
Added elements
If a new element is added to the schema it needs to either be optional, dynamic, or have a default
value. New elements with a default are added set to their default value. New dynamic or optional
elements are simply noted as a schema change.
Re-ordering elements
An element with the same name, but in a different position on the same level, is considered to be the
same element. If its type hasn't changed it will retain its value, but if the type has changed it will be
upgraded as described below.
Type changes
If a leaf is still present but its type has changed, automatic coercions are performed, so for example
integers may be transformed to their string representation if the type changed from e.g. int32 to string.

67

CDB - The ConfD XML Database

Automatic type conversion succeeds as long as the string representation of the current value can be
parsed into its new type. (Which of course also implies that a change from a smaller integer type, e.g.
int8, to a larger type, e.g. int32, succeeds for any value - while the opposite will not hold, but might!)
If the coercion fails, any supplied default value will be used. If no default value is present in the new
schema the automatic upgrade will fail.
Type changes when user-defined types are used are also handled automatically, provided that some
straightforward rules are followed for the type definitions. Read more about user-defined types in the
confd_types(3) manual page, which also describes these rules.
Hash changes
When a hash value of particular element has changed (due to an addition of, or a change to, a
tailf:id-value statement) CDB will update that element.
Key changes
When a key of a list is modified, CDB tries to upgrade the key using the same rules as explained above
for adding, deleting, re-ordering, change of type, and change of hash value. If automatic upgrade of
a key fails the entire list instance will be deleted.
Default values
If a leaf has a default value, which has not been changed from its default, then the automatic upgrade
will use the new default value (if any). If the leaf value has been changed from the old default, then
that value will be kept.
Adding / Removing namespaces
If a namespace no longer is present after an upgrade, CDB removes all data in that namespace. When
CDB detects a new namespace, it is initialized with default values.
Changing to/from operational
Elements that previously had config false set that are changed into database elements will be
treated as a added elements. In the opposite case, where data elements in the new data model are
tagged with config false, the elements will be deleted from the database.
Callpoint changes
CDB only considers the part of the data model in YANG modules that do not have external callpoints
(see Chapter 7, The external database API). But while upgrading, CDB does handle moving subtrees
into CDB from a callpoint and vice versa. CDB simply considers these as added and deleted schema
elements.
Thus an application can be developed using CDB in the first development cycle. When the external
database component is ready it can easily replace CDB without changing the schema.
Should the automatic upgrade fail, exit codes and log-entries will indicate the reason (see Section 28.11,
“Disaster management”).

5.10. Using initialization files for upgrade
As described earlier, when ConfD starts with an empty CDB database, CDB will load all instantiated XML
documents found in the CDB directory and use these to initialize the the database. We can also use this
mechanism for CDB upgrade, since CDB will again look for files in the CDB directory ending in .xml
when doing an upgrade.
This allows for handling many of the cases that the automatic upgrade can not do by itself, e.g. addition
of mandatory leaves (without default statements), or multiple instances of new dynamic containers. Most

68

CDB - The ConfD XML Database

of the time we can probably simply use the XML init file that is appropriate for a fresh install of the new
version also for the upgrade from a previous version.
When using XML files for initialization of CDB, the complete contents of the files is used. On upgrade
however, doing this could lead to modification of the user's existing configuration - e.g. we could end up
resetting data that the user has modified since CDB was first initialized. For this reason two restrictions
are applied when loading the XML files on upgrade:
• Only data for elements that are new as of the upgrade (i.e. elements that did not exist in the previous
schema) will be considered.
• The data will only be loaded if all old (i.e. previously existing) optional/dynamic parent elements and
instances exist in the current configuration.
To clarify this, we will look again at Example 5.1, “a simple server data model, servers.yang”. In
version 1.5 of the server manager, it was realized that the data model had a serious shortcoming: There was
no way to specify the protocol to use, TCP or UDP. To fix this, another leaf was added to the /servers/
server list, and the new YANG module looks like this:

Example 5.10. Version 1.5 of the servers.yang module
module servers {
namespace "http://example.com/ns/servers";
prefix servers;
import ietf-inet-types {
prefix inet;
}
revision "2007-06-01" {
description "added protocol.";
}
revision "2006-09-01" {
description "Initial servers data model";
}
/* A set of server structures
container servers {
list server {
key name;
max-elements 64;
leaf name {
type string;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
leaf protocol {
type enumeration {
enum tcp;
enum udp;
}
mandatory true;

*/

69

CDB - The ConfD XML Database

}
}
}
}

Since it was considered important that the user explicitly specified the protocol, the new leaf was made
mandatory. Of course the XML init file was updated to include this leaf, and now looks like this:


www
192.168.3.4
88
tcp


www2
192.168.3.5
80
tcp


smtp
192.168.3.4
25
tcp


dns
192.168.3.5
53
udp



We can then just use this new init file for the upgrade, and the existing server instances in the user's
configuration will get the new /servers/server/protocol leaf filled in as expected. However
some users may have deleted some of the original servers from their configuration, and in those cases we
obviously do not want those servers to get re-created during the upgrade just because they are present in
the XML file - the above restrictions make sure that this does not happen. Here is what the configuration
looks like after upgrade if the "smtp" server has been deleted before upgrade:


dns
192.168.3.5
53
udp


www
192.168.3.4
88
tcp


www2
192.168.3.5
80
tcp

70

CDB - The ConfD XML Database




This example also implicitly shows a limitation with this method: If the user has created additional servers,
the new XML file will not specify what protocol to use for those servers, and the upgrade cannot succeed
unless the external program method is used, see below. However the example is a bit contrived - in
practice this limitation is rarely a problem: It does not occur for new lists or optional elements, nor for
new mandatory elements that are not children of old lists. And in fact correctly adding this "protocol" leaf
for user-created servers would require user input - it can not be done by any fully automated procedure.

Note
Since CDB will attempt to load all *.xml files in the CDB directory at the time of upgrade, it is
important to not leave XML init files from a previous version that are no longer valid there.
It is always possible to write an external program to change the data before the upgrade transaction is
committed. This will be explained in the following sections.

5.11. Using MAAPI to modify CDB during
upgrade
To take full control over the upgrade transaction, ConfD must be started using the --startphase{0,1,2} command line options. When ConfD is started using the --start-phase0 option
CDB will initiate, and if it detects an upgrade situation the upgrade transaction will be created, and all
automatic upgrades will be performed. After which ConfD simply waits, either for a MAAPI connection
or confd --start-phase1.
Whenever changes to the schema cannot be handled automatically, or when the application programmer
wants more control over how the data in the upgraded database is populated it is possible to use MAAPI
to attach and write to the upgrade transaction in progress (see Chapter 23, The Management Agent API
for details on this API).
Using the maapi_attach_init() function call an external program can attach to the upgrade
transaction during phase0. For example a program that creates the optional container edible on each
flower in the previous forest example would look like this:

Example 5.11. Writing to an upgrade transaction using MAAPI
int th;
maapi_attach_init(ms, &th);
maapi_set_namespace(ms, th, simple__ns);
maapi_init_cursor(sock, th, &mc, "/forest/flower");
maapi_get_next(&mc);
while (mc.n != 0) {
maapi_create(sock, th, "/forest/flower{%x}/edible", &mc.keys[0]);
maapi_get_next(&mc);
}
maapi_destroy_cursor(&mc);
exit(0);

Note the use of the special maapi_attach_init() function, it attaches the MAAPI socket to the
upgrade transaction (or init transaction) and returns (through the second argument) the transaction handle

71

CDB - The ConfD XML Database

we need to make further MAAPI calls. This special upgrade transaction is only available during phase0.
Once we call confd --start-phase1 the transaction will be committed.
This method can also be combined with the init file usage described in the previous section - the data from
the init file will be applied immediately following the automatic conversions at the beginning of phase0,
and the external program can then use MAAPI to modify or complement the result.

5.12. More complex schema upgrades
In the previous section we showed how to use MAAPI to access the upgrade transaction after the automatic
upgrade had taken place. But this means that CDB has deleted all values that are no longer part of the
schema. Well, not quite yet. In start-phase0 it is possible to use all the CDB C-API calls to access the
data using the schema from the database as it looked before the automatic upgrade. That is, the complete
database as it stood before the upgrade is still available to the application. This allows us to write programs
that transfer data in an application specific way between software releases.
Say, for example, that the developers of the Example 5.1, “a simple server data model, servers.yang”
now has decided that having all servers under one top-element is not enough for their 2.0 release. They
want to instead have different categories of servers under different server types. Also a new IP address
is added to each server, the adminIP. The new version of servers.yang could then look like this
(version 1.5 has not been merged to the 2.0 branch yet):

Example 5.12. Version 2 of the servers.yang module
module servers {
namespace "http://example.com/ns/servers";
prefix servers;
import ietf-inet-types {
prefix inet;
}
revision "2007-07-15" {
description "Split servers into www and others";
}
revision "2006-09-01" {
description "Initial servers data model";
}
container servers {
list www {
key name;
max-elements 32;
leaf name {
type string;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
leaf adminIP {

72

CDB - The ConfD XML Database

type inet:ip-address;
mandatory true;
}
}
list others {
key name;
max-elements 32;
leaf name {
type string;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
leaf adminIP {
type inet:ip-address;
mandatory true;
}
}
}
}

The plan for upgrading users with the old version of the software is to transfer all servers with their name
containing the letters "www" or whose port is equal to 80, to the /servers/www subtree, and all the
others to the /servers/others subtree. The new adminIP element will be initialized to 10.0.0.1 for
the first server, and then increased by one for each server found in the old database. The code to perform
this operation would have to be written like this:

Example 5.13. The upgrade() function of server_upgrade.c
static struct sockaddr_in addr; /* Keeps address to confd daemon */
/* confd_init() must be called before calling this function */
/* ms and cs are assumed to be valid sockets */
static void upgrade(int ms, int cs)
{
int th;
int i, n;
static struct in_addr admin_ip;
cdb_connect(cs, CDB_READ_SOCKET,
(struct sockaddr *)&addr, sizeof(addr));
cdb_start_session(cs, CDB_RUNNING);
cdb_set_namespace(cs, servers__ns);
maapi_connect(ms, (struct sockaddr *)&addr, sizeof(addr));
maapi_attach_init(ms, &th);
maapi_set_namespace(ms, th, servers__ns);
admin_ip.s_addr = htonl(0x0a000001); /* initialize to 10.0.0.1 */
n = cdb_num_instances(cs, "/servers/server");
printf("servers = %d\n", n);
for (i=0; i < n; i++) {
char name[128];
confd_value_t ip, port, aip;

73

CDB - The ConfD XML Database

char *dst;
/* read old database using cdb_* API */
cdb_get_str(cs, name, 128, "/servers/server[%d]/name", i);
cdb_get(cs, &ip,
"/servers/server[%d]/ip", i);
cdb_get(cs, &port, "/servers/server[%d]/port", i);
if ((CONFD_GET_UINT16(&port) == 80) ||
(strstr(name, "www") != NULL)) {
dst = "/servers/www{%s}";
} else {
dst = "/servers/others{%s}";
}
/* now create entries in the new database using maapi */
maapi_create(ms, th, dst, name);
maapi_pushd(ms, th, dst, name);
maapi_set_elem(ms, th, &ip, "ip");
maapi_set_elem(ms, th, &port, "port");
CONFD_SET_IPV4(&aip, admin_ip);
maapi_set_elem(ms, th, &aip, "adminIP");
admin_ip.s_addr = htonl(ntohl(admin_ip.s_addr) + 1);
maapi_popd(ms, th);
}
cdb_end_session(cs);
cdb_close(cs);
}

It would of course be wise to add error checks to the code above. Also note that choosing new values for
added elements can be done in a number of different ways, perhaps the end-user needs to be prompted,
perhaps the data resides elsewhere on the device.
Now assuming the data in CDB is as in the "Initialization data for CDB" figure in the section above, then
running the upgrade code would be done in the following order.
$ confd --stop
... Install new versions of software and fxs files ...
$ confd --start-phase0
$ server_upgrade
$ confd --start-phase1
... Start other internal daemons ...
$ confd --start-phase2

Finally, after ConfD is up and running after the upgrade the data would look like this.


www
192.168.3.4
88
10.0.0.3


www2
192.168.3.5
80
10.0.0.4


dns

74

CDB - The ConfD XML Database

192.168.3.5
53
10.0.0.1


smtp
192.168.3.4
25
10.0.0.2



ConfD does not impose any specific meaning to "version" - any change in the data model is an upgrade
situation as far as CDB is concerned. ConfD also does not force the application programmer to handle
software releases in a specific way, as each application may have very different needs and requirements
in terms of footprint and storage etc.

5.13. The full dhcpd example
As an example, we show how to integrate dhcpd - the ISC DHCP daemon - under ConfD.
Assume we have Example 5.14, “A YANG module describing a dhcpd server configuration” in a file
called dhcpd.yang.

Example 5.14. A YANG module describing a dhcpd server configuration
module dhcpd {
namespace "http://example.com/ns/dhcpd";
prefix dhcpd;
import ietf-inet-types {
prefix inet;
}
import tailf-xsd-types {
prefix xs;
}
typedef loglevel {
type enumeration {
enum kern;
enum mail;
enum local7;
}
}
grouping subNetworkType {
leaf net {
type inet:ip-address;
}
leaf mask {
type inet:ip-address;
}
container range {
presence "";
leaf dynamicBootP {
type boolean;
default false;
}

75

CDB - The ConfD XML Database

leaf lowAddr {
type inet:ip-address;
mandatory true;
}
leaf hiAddr {
type inet:ip-address;
}
}
leaf routers {
type string;
}
leaf maxLeaseTime {
type xs:duration;
default PT7200S;
}
}
container dhcp {
leaf defaultLeaseTime {
type xs:duration;
default PT600S;
}
leaf maxLeaseTime {
type xs:duration;
default PT7200S;
}
leaf logFacility {
type loglevel;
default local7;
}
container SubNets {
container subNet {
uses subNetworkType;
}
}
container SharedNetworks {
list sharedNetwork {
key name;
max-elements 1024;
leaf name {
type string;
}
container SubNets {
container subNet {
uses subNetworkType;
}
}
}
}
}
}

We use some interesting constructs in this module. We define a grouping and reuse it twice in the
module. This is what we do when we want to reuse a data model structure defined somewhere else in
the specification.
Now we have a data model which is fully usable with ConfD. Consider an actual dhcpd.conf file which
looks like:
defaultleasetime 600;
maxleasetime 7200;

76

CDB - The ConfD XML Database

subnet 192.168.128.0 netmask 255.255.255.0 {
range 192.168.128.60 192.168.128.98;
}
sharednetwork 22429 {
subnet 10.17.224.0 netmask 255.255.255.0 {
option routers rtr224.example.org;
}
subnet 10.0.29.0 netmask 255.255.255.0 {
option routers rtr29.example.org;
}
}

The above dhcp configuration would be represented by an XML structure that looks like this:

7200
600


192.168.128.0
255.255.255.0

192.168.128.60
192.168.128.98





22429


10.17.224.0
255.255.255.0
rtr224.example.org


10.0.29.0
255.255.255.0
rtr29.example.org






The dhcp server subscribes to configuration changes and reconfigures when notified. For our purposes we
write a small program which:
1. Reads the configuration database.
2. Writes /etc/dhcp/dhcpd.conf and HUPs the daemon.
The main function looks exactly like the example where we read the servers database with the exception
that we establish a subscription socket for the path /dhcp instead of /servers. Whenever any
configuration change occurs for anything related to dhcp, the program rereads the configuration from CDB,
regenerates the dhcpd.conf file and HUPs the dhcp daemon.
Reading the dhcp configuration from CDB is done as follows:

77

CDB - The ConfD XML Database

static int read_conf(struct sockaddr_in *addr)
{
FILE *fp;
struct confd_duration dur;
int i, n, tmp;
int rsock;
if ((rsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open socket\n");
if (cdb_connect(rsock, CDB_READ_SOCKET, (struct sockaddr*)addr,
sizeof (struct sockaddr_in)) < 0)
return CONFD_ERR;
cdb_set_namespace(rsock, dhcpd__ns);
if ((fp = fopen("dhcpd.conf.tmp", "w")) == NULL) {
cdb_close(rsock);
return CONFD_ERR;
}
cdb_get_duration(rsock, &dur, "/dhcp/defaultLeaseTime");
fprintf(fp, "default-lease-time %d\n", duration_to_secs(&dur));
cdb_get_duration(rsock, &dur, "/dhcp/maxLeaseTime");
fprintf(fp, "max-lease-time %d\n", duration_to_secs(&dur));
cdb_get_enum_value(rsock, &tmp, "/dhcp/logFacility");
switch (tmp) {
case dhcpd_kern:
fprintf(fp, "log-facility kern\n");
break;
case dhcpd_mail:
fprintf(fp, "log-facility mail\n");
break;
case dhcpd_local7:
fprintf(fp, "log-facility local7\n");
break;
}
n = cdb_num_instances(rsock, "/dhcp/subNets/subNet");
for (i=0; i 0) {
if ((status = read_conf(&addr)) != CONFD_OK) {
fprintf(stderr, "Terminate: read_conf %d\n", status);
exit(1);
}
}
rename("dhcpd.conf.tmp", "/etc/dhcpd.conf");
system("killall -HUP dhcpd");
if ((status = cdb_sync_subscription_socket(subsock,

80

CDB - The ConfD XML Database

CDB_DONE_PRIORITY))
!= CONFD_OK) {
fprintf(stderr, "failed to sync subscription: %d\n", status);
exit(1);
}
}

81

Chapter 6. Operational Data
6.1. Introduction to Operational Data
In Chapter 3, The YANG Data Modeling Language we showed how to define data models in YANG. In
Chapter 5, CDB - The ConfD XML Database we showed how to use CDB and also how to interface CDB
to external daemons. In this chapter, we show how to write instrumentation code for read-only operational
and statistics data.
Operational data is typically not kept in a database but read at runtime by instrumentation functions. This
would for example be statistics counters contained inside the managed objects themselves. In this chapter
we will show how to write such instrumentation functions in C.
An alternative approach to runtime operational data is to store the operational in CDB using a write
interface. This will be described in Section 6.8, “Operational data in CDB”.
The configuration of the network device is modeled by a YANG module. This describes the data model
of the device. We also need to write YANG modules for our operational data.
In the YANG data model, there can be restrictions on valid operational data. For example, a list might
have a "max-elements" constraint, or a "must" expression associated with it. For performance reasons,
ConfD does not check these constraints. It is assumed that the application code that generates operational
data enforces the constraints.
Normally, operational data is strictly read-only. If the operational state of the device needs to be modified, it
is typically done through special operations (rpc or actions in NETCONF, or special commands in the CLI).
But this imposes a problem with protocols like SNMP, that do not have a mechanism to invoke arbitrary
operations. In SNMP, this is solved by writing values to special objects, called writable operational objects.
These objects are implemented in the same way as writable configuration data, described in Section 7.8,
“Writable operational data”, and the section called “Writable MIB objects”.

6.2. Reading Statistics Data
A very common situation is that we wish to expose statistics data from the device. Consider for example
the output of the netstat -i command.
#root netstat -i
Iface MTU
Met
RX-OK RX-ERR RX-DRP TX-OK TX-ERR TX-DRP Flg
eth0
1500
0 212684
0
0 142470
0
0 BMRU
lo
16436 0
2077
0
0
2077
0
0 LRU

This is useful information to expose to the Web UI, the CLI or a management application running
NETCONF.
To address this we must do two things; the statistics information must be modeled in a YANG module:

Example 6.1. netstat.yang
container ifaces {
config false;
list iface {
key name;
max-elements 1024;
leaf name {

82

Operational Data

type string;
}
leaf mtu {
type uint32;
}
leaf metric {
type uint64;
}
leaf rx_ok {
type uint64;
}
leaf rx_err {
type uint64;
}
leaf rx_drp {
type uint64;
}
leaf tx_ok {
type uint64;
}
leaf tx_err {
type uint64;
}
leaf tx_drop {
type uint64;
}
leaf flag {
type string;
}
}
}

The above simple one-to-one mapping of the netstat -i output and a YANG data model might suffice for
our needs. It can be refined later.
The second thing that must be done is to write C code that parses the netstat -i output. Finally we must
connect that C code to ConfD. That procedure will be fully described in this chapter.
Thus we need to:
• Write a YANG module describing our operational data (see Chapter 3, The YANG Data Modeling
Language).
• Write a mapping between the data model and the operational data as represented on the target device.
The mapping is specified inside the data model itself, using callbacks to C.

6.3. Callpoints and Callbacks
The data model indicates where to invoke callbacks by annotation with callpoints. A callpoint has a
name which later can be used by an external program to connect to that named point.

Tip
We can always define callpoints in a separate YANG module by using the tailf:annotate
extension as described in the tailf_yang_extensions(5) manual page. This way we can keep the
data model free from implementation specific details.
Assume that we wish to model the ARP table of the host:

83

Operational Data

Example 6.2. ARP table YANG module
module arpe {
namespace "http://tail-f.com/ns/example/arpe";
prefix arpe;
import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
container arpentries {
config false;
tailf:callpoint arpe;
list arpe {
key "ip ifname";
max-elements 1024;
leaf ip {
type inet:ip-address;
}
leaf ifname {
type string;
}
leaf hwaddr {
type string;
mandatory true;
}
leaf permanent {
type boolean;
mandatory true;
}
leaf published {
type boolean;
mandatory true;
}
}
}
}

The arpe callpoint will invoke callbacks in external programs that has registered itself with the name
"arpe". The programs use the API in the libconfd.so library to register themselves under different
callpoints.
The config false; statement instructs ConfD that the entire arpentries container is nonconfiguration data. Data below that point is not part of the configuration; rather it should be viewed as
ephemeral read-only data.
Assume we have the above YANG module loaded in ConfD. Furthermore that ConfD receives a
NETCONF "get" request like:




ConfD is configured to accept a number of arpe list entries contained inside an arpentries container.
It does not know which arpe entries reside on the device though. With the above NETCONF request, the
task for ConfD is to produce an XML structure containing all the arpe entries on the device.

84

Operational Data

This is solved by letting the application register itself with a set of callback C functions under the callpoint.
The callback C functions do things like get_next(), get_elem() and so forth.
There can be several different C programs on the same device which register themselves under different
callpoints. These C programs that register with ConfD are referred to as daemons.

Daemons using libconfd.so to connect to ConfD.
In the above picture we show how two separate C programs (daemons) connect to ConfD using the
libconfd.so shared library.

6.4. Data Callbacks
Each callpoint in a YANG module must have an associated set of callback functions. The following data
callback functions are required for operational data:
get_next()
This callback is invoked repeatedly to find out which keys exist for a certain list. ConfD will invoke
the callback as a means to iterate through all entries of the list, in this case all arpe entries. For
example, assume that the ARP table on the device looks as:

Example 6.3. Populated ARP table



85

Operational Data

192.168.1.1
eth0
00:30:48:88:1F:E2
false
false


192.168.1.42
eth0
00:30:48:88:1F:C5
false
false



The job of the get_next() callback would be to return the first key on the first invocation, namely
the pair "192.168.1.1", "eth0" and then subsequently the remaining keys until there are no more keys.
(The data model says that we have two keys, ip and ifname.)
get_elem()
This callback is invoked by ConfD when ConfD needs to read the actual value of a leaf element. We
must also implement the get_elem() callback for the keys. ConfD invokes get_elem() on a
key as an existence test.
exists_optional()
This callback is called for all typeless and optional elements, i.e. presence containers and leafs of
type empty. For example the YANG module fragment:
container bs {
presence "bs";
config false;
tailf:callpoint bcp;
leaf foo {
type string;
}
}

If we do not have any typeless optional elements in our data model we need not implement this callback
and can set it to NULL. A detailed description of this callback can be found in the confd_lib_dp(3)
manual page.
We also have a number of additional optional callbacks that may be implemented for efficiency reasons.
The precise usage of these optional callbacks is described in the man page confd_lib_dp(3).
get_object()
If this optional callback is implemented, the work of the callback is to return an entire object, i.e.
a list entry. In this case all the five elements contained in an arpe entry - namely the ip, ifname,
hwaddr, permanent and finally published leafs.
num_instances()
When ConfD needs to figure out how many entries we have for a list, by default ConfD will repeatedly
invoke the get_next() callback. If this callback is registered, it will be called instead.
get_next_object()
This optional callback combines get_next() and get_object() into a single callback. This
callback only needs to be implemented when it is very important to be able to traverse a table fast.

86

Operational Data

find_next()
This callback primarily optimizes cases where ConfD wants to start a list traversal at some other point
than at the first entry of the list. It is mainly useful for lists with a large number of entries. If it is not
registered, ConfD will use a sequence of get_next() calls to find the desired list entry.
find_next_object()
This callback combines find_next() and get_object() into a single callback.

6.5. User Sessions and ConfD Transactions
In this section we will describe a number of new concepts. We will define what we mean by a user session
and what ConfD transactions are. This will be further explained in Chapter 7, The external database API.
A user session corresponds directly to an SSH/SSL session from a management station to ConfD. A user
session is associated with such data as the IP address of the management station and the user name of the
user who started the session, independent of northbound agent.
The user session data is always available to all callback functions.
A new transaction is started whenever an agent tries to read operational data. For each transaction two
user defined callbacks are potentially invoked:
init()

From the daemon's point of view, this callback will be invoked when a transaction starts.
However as an optimization, ConfD will delay the invocation for a given daemon until
the point where some data needs to be read, i.e. just before the first get_next(),
get_elem(), etc callback.

finish()

This callback gets invoked at the end of the transaction, if init() has been invoked. This
is a good place to deallocate any local resources for the transaction. This callback is optional.

The "lazy" invocation of init() means that for a transaction where none of the operational data provided
by a given daemon is accessed, that daemon will not have any callbacks at all invoked.

6.6. C Example with Operational Data
Assume we want to provide the state of the current ARP table on the device. To do this we need to write a
YANG module which models an ARP table, and then write C functions which populates the corresponding
XML tree. We use the YANG module from the previous section and save it to a file arpe.yang and
compile the module using the confdc compiler as:
# confdc -c arpe.yang
# confdc --emit-h arpe.h arpe.fxs

The --emit-h option to confdc is used to generate a header file. Thus, in our example the generated file
will be called arpe.h. The generated header file contains a mapping from the strings found in the data
model such as ip or permanent to integer values.
Finally we must instruct ConfD where to find the newly generated schema file. Using the default ConfD
configuration, ConfD looks for schema (.fxs) files under /etc/confd:
# cp arpe.fxs /etc/confd
# confd

87

Operational Data

After loading arpe.fxs, ConfD runs with the newly generated data model. Next we need to write the C
program which provides the ARP data by means of C callback functions.
An actual running version of this example can be found in the intro/5-c_stats directory in the
examples in the distribution release. We will walk through this C program here.
First we need to include confd_lib.h and confd_dp.h which are part of a ConfD release, as well
as the newly generated arpe.h. See confdc (1) for details.
#include 
#include 
#include "arpe.h"

We use a couple of global variables as well as a structure which represents an ARP entry.
/* Our
static
static
static

daemon context as a global variable */
struct confd_daemon_ctx *dctx;
int ctlsock;
int workersock;

struct aentry {
struct in_addr ip4;
char *hwaddr;
int perm;
int pub;
char *iface;
struct aentry *next;
};
struct arpdata {
struct aentry *arp_entries;
struct timeval lastparse;
};

The struct confd_daemon_ctx *dctx is a daemon context. It is a data structure which is passed to virtually
all the functions.
We are ready for the main() function. There we will initialize the library, connect to the ConfD daemon
and install a number of callback functions as pointers to C functions. Remember the architecture of this
system, ConfD executes as a common daemon, and the program we are writing executes outside the address
space of ConfD. Our program links with the ConfD library (libconfd.so) which manages the protocol
between our application and ConfD.
int main(int argc, char *argv[])
{
struct sockaddr_in addr;
int debuglevel = CONFD_TRACE;
struct confd_trans_cbs trans;
struct confd_data_cbs data;
memset(&trans, 0, sizeof (struct confd_trans_cbs));
trans.init = s_init;
trans.finish = s_finish;
memset(&data, 0, sizeof (struct confd_data_cbs));
data.get_elem = get_elem;
data.get_next = get_next;

88

Operational Data

strcpy(data.callpoint, arpe__callpointid_arpe);
/* initialize confd library */
confd_init("arpe_daemon", stderr, debuglevel);
addr.sin_addr.s_addr = inet_addr("127.0.0.1");
addr.sin_family = AF_INET;
addr.sin_port = htons(CONFD_PORT);
if (confd_load_schemas((struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) != CONFD_OK)
confd_fatal("Failed to load schemas from confd\n");
if ((dctx = confd_init_daemon("arpe_daemon")) == NULL)
confd_fatal("Failed to initialize confdlib\n");
/* Create the first control socket, all requests to */
/* create new transactions arrive here */
if ((ctlsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open ctlsocket\n");
if (confd_connect(dctx, ctlsock, CONTROL_SOCKET, (struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");
/* Also establish a workersocket, this is the most simple */
/* case where we have just one ctlsock and one workersock */
if ((workersock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open workersocket\n");
if (confd_connect(dctx, workersock, WORKER_SOCKET,(struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");
if (confd_register_trans_cb(dctx, &trans) == CONFD_ERR)
confd_fatal("Failed to register trans cb \n");
if (confd_register_data_cb(dctx, &data) == CONFD_ERR)
confd_fatal("Failed to register data cb \n");
if (confd_register_done(dctx) != CONFD_OK)
confd_fatal("Failed to complete registration \n");

At this point we have registered our callback functions for data manipulations under the arpe callpoint.
Whenever data needs to manipulated below that callpoint our C callback functions should be invoked.
The confd_register_done() call tells ConfD that we are done with the callback registrations - no
callbacks will be invoked before we issue this call.
The arpe__callpointid_arpe symbol that is used for the callpoint element in the data callback
registration is one of the definitions in the generated arpe.h file. It just maps to the string "arpe"
that we could have used instead, but by using the symbol we make sure that if the name given with the
tailf:callpoint statement in the YANG module is changed, without a corresponding change in the
C code, the problem is detected at compile time.
We have also created one control socket and one worker socket. These are sockets owned by the application
and they should be added to the poll() or select() set of the application.
All new requests that arrive from ConfD arrive on the control socket. As we will see, the init()
callback must call the API function confd_trans_set_fd() which will assign a worker socket to

89

Operational Data

the transaction. All further requests and replies for this transaction will be sent on the worker socket. We
can have several worker sockets and they can run in different operating system threads than the thread
owning the control socket.
The poll loop could look like:
while(1) {
struct pollfd set[2];
int ret;
set[0].fd = ctlsock;
set[0].events = POLLIN;
set[0].revents = 0;
set[1].fd = workersock;
set[1].events = POLLIN;
set[1].revents = 0;
if (poll(set, sizeof(set)/sizeof(*set), -1) < 0) {
perror("Poll failed:");
continue;
}
/* Check for I/O */
if (set[0].revents & POLLIN) {
if ((ret = confd_fd_ready(dctx, ctlsock)) == CONFD_EOF) {
confd_fatal("Control socket closed\n");
} else if (ret == CONFD_ERR && confd_errno != CONFD_ERR_EXTERNAL) {
confd_fatal("Error on control socket request: %s (%d): %s\n",
confd_strerror(confd_errno), confd_errno, confd_lasterr());
}
}
if (set[1].revents & POLLIN) {
if ((ret = confd_fd_ready(dctx, workersock)) == CONFD_EOF) {
confd_fatal("Worker socket closed\n");
} else if (ret == CONFD_ERR && confd_errno != CONFD_ERR_EXTERNAL) {
confd_fatal("Error on worker socket request: %s (%d): %s\n",
confd_strerror(confd_errno), confd_errno, confd_lasterr());
}
}
}

The crucial function above is confd_fd_ready(). When either of the (in this case, two) sockets
from the application to ConfD are ready to read, the application is responsible for invoking the
confd_fd_ready() function. This function will read data from the socket, unmarshal that data and
invoke the right callback function with the right arguments.
We have installed two transaction callbacks: init() and finish(), and also two data callbacks:
get_next() and get_elem().
The two transaction callbacks look like:
static int s_init(struct confd_trans_ctx *tctx)
{
struct arpdata *dp;
if ((dp = malloc(sizeof(struct arpdata))) == NULL)
return CONFD_ERR;

90

Operational Data

memset(dp, 0, sizeof(struct arpdata));
if (run_arp(dp) == CONFD_ERR) {
free(dp);
return CONFD_ERR;
}
tctx->t_opaque = dp;
confd_trans_set_fd(tctx, workersock);
return CONFD_OK;
}
static int s_finish(struct confd_trans_ctx *tctx)
{
struct arpdata *dp = tctx->t_opaque;
if (dp != NULL) {
free_arp(dp);
free(dp);
}
return CONFD_OK;
}

The init() callback reads the ARP table calling a function run_arp() and stores a local copy of a
parsed ARP table in the transaction context. This data structure (struct confd_trans_ctx *tctx) is allocated
by the library and used throughout the entire transaction. The t_opaque field in the transaction context
is meant to be used by the application to store transaction local data.
A naive version of run_arp() could call popen(3) on the command arp -an and parse the output:
# arp -an
? (192.168.128.33) at 00:40:63:C9:79:FC [ether] on eth1
? (217.209.73.1) at 00:02:3B:00:3B:67 [ether] on eth0

The parsed ARP table created by run_arp() is ordered by increasing key values, since ConfD expects
us to return entries in that order when traversing the list.
There may be several ConfD transactions running in parallel and some transactions may have been initiated
from the CLI and the current ARP data may be stale or may be nonexistent.
The init() callback must also indicate to the library which socket should be used for all future traffic for
this transaction. In our case, we have just one option, namely the single worker socket we created. This is
done through the call to confd_trans_set_fd(). Also, the init() callback was fed a transaction
context parameter. This structure is allocated by the library and fed to each and every callback function
executed during the life of the transaction. The structure is defined in confd_lib.h.
Our finish() function cleans up everything.
The data callbacks look like:
static int get_next(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath,
long next)
{
struct arpdata *dp = tctx->t_opaque;
struct aentry *curr;
confd_value_t v[2];
if (next == -1) { /* first call */
if (need_arp(dp)) {

91

Operational Data

if (run_arp(dp) == CONFD_ERR)
return CONFD_ERR;
}
curr = dp->arp_entries;
} else {
curr = (struct aentry *)next;
}
if (curr == NULL) {
confd_data_reply_next_key(tctx, NULL, -1, -1);
return CONFD_OK;
}
/* 2 keys */
CONFD_SET_IPV4(&v[0], curr->ip4);
CONFD_SET_STR(&v[1], curr->iface);
confd_data_reply_next_key(tctx, &v[0], 2, (long)curr->next);
return CONFD_OK;
}

struct aentry *find_ae(confd_hkeypath_t *keypath, struct arpdata *dp)
{
struct in_addr ip = CONFD_GET_IPV4(&keypath->v[1][0]);
char *iface = (char*)CONFD_GET_BUFPTR(&keypath->v[1][1]);
struct aentry *ae = dp->arp_entries;
while (ae != NULL) {
if (ip.s_addr == ae->ip4.s_addr &&
(strcmp(ae->iface, iface) == 0) )
return ae;
ae=ae->next;
}
return NULL;
}
/* Keypath example */
/* /arpentries/arpe{192.168.1.1 eth0}/hwaddr */
/*
3
2
1
0
*/
static int get_elem(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
confd_value_t v;
struct aentry *ae = find_ae(keypath, tctx->t_opaque);
if (ae == NULL) {
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
switch (CONFD_GET_XMLTAG(&(keypath->v[0][0]))) {
case arpe_hwaddr:
if (ae->hwaddr == NULL) {
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
CONFD_SET_STR(&v, ae->hwaddr);
break;
case arpe_permanent:
CONFD_SET_BOOL(&v, ae->perm);
break;

92

Operational Data

case arpe_published:
CONFD_SET_BOOL(&v, ae->pub);
break;
case arpe_ip:
CONFD_SET_IPV4(&v, ae->ip4);
break;
case arpe_ifname:
CONFD_SET_STR(&v, ae->iface);
break;
default:
return CONFD_ERR;
}
confd_data_reply_value(tctx, &v);
return CONFD_OK;
}

The above code needs a bit of explaining. Before doing this we need to look at how the confd_hkeypath_t
data type works.
All the different data manipulation callbacks get a hashed keypath as a parameter. For example when a
daemon gets invoked in get_elem() and ConfD wants to read the published element for a specific
arp entry, the textual representation of the hkeypath is /arpentries/arpe{1.2.3.4 eth0}/
published.
The C representation of a hashed keypath is a fixed size array of values, as in:
typedef struct confd_hkeypath {
confd_value_t v[MAXDEPTH][MAXKEYLEN];
int len;
} confd_hkeypath_t;

The keypath is fed in the reverse order to the application, thus - when ConfD wants to read /
arpentries/arpe{1.2.3.4 eth0}/published, the following holds for the keypath:
• keypath->v[0][0] is the XML element, namely published.
• keypath->v[1][0] is the first key one step up, in our case the IP address 1.2.3.4.
• keypath->v[1][1] is the second key one step up, in our case the interface name eth0.
• keypath->v[2][0] is the XML element two steps up, namely arpe.
• keypath->v[3][0] is the XML element three steps up, namely arpentries. The top level
element. This item could also have been obtained through the expression keypath->v[keypath>len - 1][0].
The actual values are represented as a union struct defined in confd_lib.h. The confd_value_t data
type can represent all ground data types such as strings, integers, but also slightly more complex data types
such as IP addresses and the various date and time data types found in XML schema.
confd_lib.h defines a set of macros to set and get the actual values from confd_value_t variables. For
example this code sets and gets an individual value:
confd_value_t myval;
int i = 99;
CONFD_SET_INT32(&myval, i);

93

Operational Data

assert(99 == CONFD_GET_INT32(&myval));

One important variant of confd_value_t is string. All data values which are of type string, or of a
type derived from string, are passed from ConfD to the application as NUL terminated strings. Thus
confd_value_t contains a length indicator and is NUL terminated.
All strings consist of an unsigned char* pointer and a length indicator. To copy such a string into a local
buffer we need to write code like:
char *mybuf = malloc(CONFD_GET_BUFSIZE(someval)+1);
strcpy(mybuf, (char*)CONFD_GET_BUFPTR(someval));

On the other hand, when the application needs to reply with a string value to ConfD, the application can
choose to use either a NUL terminated string or a buffer with a length indicator using the following macros:
confd_value_t myval;
CONFD_SET_STR(&myval, "Frank Zappa");

or
confd_value_t myval;
CONFD_SET_BUF(&myval, buf, buflen);

XML tags are also represented as confd_value_t. Remember that we said that keypath->v[0][0] was
the actual XML element. Also remember that confdc generated a .h file. The arpe.h file, containing all
the XML elements from arpe.yang as integers. The following code uses that to switch on the XML tag:
switch (CONFD_GET_XMLTAG(&(keypath->v[0][0]))) {
case arpe_hwaddr:
if (ae->hwaddr == NULL) {
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
CONFD_SET_STR(&v, ae->hwaddr);
break;
case arpe_permanent:
CONFD_SET_BOOL(&v, ae->perm);
break;

Each keypath has a textual representation, so we can format a keypath by means of the API call
confd_pp_kpath(). A hashed keypath, a confd_hkeypath_t, represents a unique path down through
the XML tree and it is easy and efficient to walk the path through switch statements since the individual
XML elements in the path are integers.
The purpose of both functions, (get_next() and get_elem()), is to return data back to ConfD. Data
is not returned explicitly through return values from the callback functions, but rather through explicit
API calls.
So when the application gets invoked in get_elem() via a call to confd_fd_ready(), we need to
return a single value to ConfD. We do this through the call to confd_data_reply_value(). Thus
the following code snippet returns an integer value to ConfD.
confd_value_t myval;
CONFD_SET_INT32(&myval, 7777);
confd_data_reply_value(tctx, &myval);

94

Operational Data

The get_elem() callback is also used as an existence test by ConfD. It may seem redundant to
implement the get_elem() callback for a keypath such as: "/arpentries/arpe{1.2.3.4
eth0}/ip" since the only possible reply can be the IP address "1.2.3.4" which is already part of the
keypath. However, the user can enter any random path in the CLI and ConfD uses the get_elem()
callback to check whether an entry exists or not.
If the entry does not exist, the callback should call confd_data_reply_not_found() and then
return CONFD_OK. This is not an error.
The API is fully documented in the confd_lib_dp(3) manual page.
The get_next() callback gets invoked when ConfD needs to read all the keys of a certain list such as
our ARP entries. The next parameter will have the value -1 on the first invocation to get_next().
This invocation needs to return the first key in the ordered list created by run_arp(). In our case, with
our ARP entries, we have multiple keys. According to the data model the pair of the interface name and
the IP address makes up the key. Thus we need to return two values:
CONFD_SET_IPV4(&v[0], curr->ip4);
CONFD_SET_STR(&v[1], curr->iface);
confd_data_reply_next_key(tctx, &v[0], 2, (long)curr->next);

The last parameter to confd_data_reply_next_key() is a long integer which will be fed to us as
the next parameter on the subsequent call. We cast the pointer to the next struct aentry* as a long.
In the above code, we registered a single set of callback C functions on the callpoint. Sometimes we
may have different daemons that handle different kinds of data, but under the same callpoint. Say for
example that we have a list of interfaces, VLAN interfaces and regular interfaces. We have different
software modules which handle the VLAN interfaces and the regular interfaces. In this case, we may use
confd_register_range_data_cb() (See confd_lib_dp(3)) which makes it possible to install a
set of callbacks on a range of keys, for example one set of callbacks for eth0 to ethX and another set
of callbacks in the range from vlan0 to vlanX.
Also notable is the consistency of the data. If we use CDB to store our configuration data and we use
this external data API to deliver statistics data which is volatile we must choose whether we want to
deliver an exact snapshot of the statistics data or not. ConfD will consecutively call get_next() to
gather all the keys for a set of dynamic elements. A moment later ConfD will invoke get_elem()
or get_object() to gather the actual data. If this data no longer exists, the application can invoke
confd_data_reply_not_found() and all is fine.
An alternative for the application if we must always return consistent snapshots, is to gather and buffer
all the data in the init() callback and then return both get_next() data as well as get_elem()
data from those internal data structures. This data can be stored in the t_opaque field in the transaction
context and be released in the finish() callback.
In our example code above we have chosen the latter approach. Also notable is the check for age of
data at the beginning of get_next(). A NETCONF transaction is typically short lived, whereas a CLI
transaction remains live for as long as the user is logged in. Thus we may have to refresh the locally stored
ARP table if it is deemed to be too old.

6.7. The Protocol and a Library Threads
Discussion
We start this section with a picture showing the sequence of events that occur when the user defined
callback functions get invoked by the library.

95

Operational Data

Event chain that triggers user callbacks
On the library side we have one control socket and one or more worker sockets. The idea behind this
architecture is that it shall be possible to have a software architecture, whereby a main thread owns the
control socket, and we also have a set of worker threads, each owning one or more worker sockets. A
request to execute something arrives from ConfD on the control socket, and the thread owning the control
socket, the main thread, can then decide to assign a worker thread for that particular activity, be it a
validation, a new transaction or the invocation of an action. The owner of the control socket must thus
have a mapping between worker sockets and thread workers. This is up to the application to decide.
The downside of the architecture proposed above is complexity, whereas the upside is that regardless of
how long time it takes to execute an individual request from ConfD, the data provider is always ready to
accept and serve new callback requests from ConfD.
The case for a multi threaded dataprovider maybe isn't as strong as one could think. Say that we have a
statistics data provider which lists a very long list of statistics items, e.g. a huge routing table. If a CLI user
invokes the command to show all routing table entries, there will be a long series of get_next() and
get_elem() callback invocations. As long as the application is still polling the control socket, other
northbound agents can very well sneak in and execute their operations while the routing table is being
displayed. For example another CLI user issuing a request to reboot the host, will get his reboot request
served at the same time as the first CLI user is displaying the large routing table.

96

Operational Data

A data provider with just one thread, one control socket and one worker socket will never hang longer than
it takes to execute a single callback invocation, e.g. a single invocation of get_elem(), validate()
or action(). In many cases it will still be a good design to use at least one thread for the control socket
and one for the worker socket - this will allow for control socket requests to be handled quickly even if the
data callbacks require more processing time. If we have long-running action callbacks (e.g. file download),
multi-threading may be essential, see Section 11.2.2, “Using Threads”.
The intro/9-c_threads example in the ConfD examples collection shows one way to use multithreading in a daemon that implements both operational data callbacks and action callbacks. It has one
thread for the control socket and only a single worker socket/thread for the data callbacks, while multiple
worker sockets/threads are used to handle the action callbacks.
When we use multiple threads, it is important to remember that threads can not "share" socket connections
to ConfD. For the data provider API, this is basically fulfilled automatically, as we will not have multiple
threads polling the same socket. But when we use e.g. the CDB or MAAPI APIs, the application must
make sure that each thread has its own sockets. I.e. the ConfD API functions are thread-safe as such, but
multiple threads using them with the same socket will have unpredictable results, just as multiple threads
using the read() and write() system calls on the same file descriptor in general will. In the ConfD case, one
thread may end up getting the response to a request from another, or even a part of that response, which
will result in errors that can be very difficult to debug.

6.8. Operational data in CDB
It is possible to use CDB to store not only the configuration data but also operational data. Depending
on the application and the underlying architecture it may be easier for some of the managed objects to
write their operational data into CDB. Depending on the type of data, this would typically be done either
at regular intervals or whenever there is a change in the data. If this is done, no instrumentation functions
need to be written. The operational data then resides in CDB and all the northbound agents can read the
operational data automatically from CDB.
Similar to the CDB read interface, we need to create a CDB socket and also start a CDB session on the
socket before we can write data
The necessary steps are:
1. cdb_connect()
2. cdb_start_session() followed by cdb_set_namespace()
3. A series of calls to one or several of the CDB set functions, cdb_set_elem(), cdb_create(),
cdb_delete() cdb_set_object() or cdb_set_values()
These functions are described in detail in the confd_lib_cdb(3) manual page.
4. A call to cdb_end_session()
It is also possible to load operational data from an XML file into CDB using the function
cdb_load_file(), see the confd_lib_cdb(3) manual page. A command line utility called confd_load
can also be used, see confd_load(1).
We use the tailf:cdb-oper statement to indicate that operational data should be stored in CDB, see
the tailf_yang_extensions(5) manual page. The data can be either persistent, i.e. stored on disc, or volatile,
i.e. stored in RAM only - this is controlled by the tailf:persistent substatement to tailf:cdboper.

97

Operational Data

As a realistic example we model IP traffic statistics in a Linux environment. We have a list of interfaces,
stored in CDB and then for each interface we have a statistics part. This example can be found in
cdb_oper/ifstatus in the examples collection. This is what our data model looks like:
module if {
namespace "http://tail-f.com/ns/example/if";
prefix if;
import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}

container interfaces {
list interface {
key name;
max-elements 1024;
leaf name {
type string;
}
list address {
key name;
max-elements 64;
leaf name {
type inet:ipv4-address;
}
leaf prefix-length {
type int32;
mandatory true;
}
}
container status {
config false;
tailf:cdb-oper;
container receive {
leaf bytes {
type uint64;
mandatory true;
}
leaf packets {
type uint64;
mandatory true;
}
leaf errors {
type uint32;
mandatory true;
}
leaf dropped {
type uint32;
mandatory true;
}
}
container transmit {
leaf bytes {
type uint64;
mandatory true;

98

Operational Data

}
leaf packets {
type uint64;
mandatory true;
}
leaf errors {
type uint32;
mandatory true;
}
leaf dropped {
type uint32;
mandatory true;
}
leaf collisions {
type uint32;
mandatory true;
}
}
}
}
}
}

Note the element /interfaces/interface/status, it has the substatement config false;
and below it we find a tailf:cdb-oper; statement. If we had implemented this operational data using
the techniques from the previous sections in this chapter, we would have had to write instrumentation
callback functions for the above operational data. For example get_elem() which would then be
given a path, e.g. /interfaces/interface{eth0}/status/receive/bytes When we use
the tailf:cdb-oper; statement these instrumentation callbacks are automatically provided internally
by ConfD. The downside is that we must populate the CDB data from the outside.
A function which reads network traffic statistics data and updates CDB according to the above data model
is:
#define GET_COUNTER() {
if ((p = strtok(NULL, " \t")) == NULL)
continue;
counter = atoll(p);
}

\
\
\
\

static int update_status(int sock)
{
FILE *proc;
int ret;
char buf[BUFSIZ];
char *ifname, *p;
long long counter;
confd_value_t val[1 + 4 + 1 + 5];
int i;
if ((ret =
return
if ((ret =
return

cdb_start_session(sock, CDB_OPERATIONAL)) != CONFD_OK)
ret;
cdb_set_namespace(sock, if__ns)) != CONFD_OK)
ret;

if ((proc = fopen("/proc/net/dev", "r")) == NULL)
return CONFD_ERR;
while (ret == CONFD_OK && fgets(buf, sizeof(buf), proc) != NULL) {
if ((p = strchr(buf, ':')) == NULL)
continue;

99

Operational Data

*p = ' ';
if ((ifname = strtok(buf, " \t")) == NULL)
continue;
i = 0;
CONFD_SET_XMLTAG(&val[i], if_receive, if__ns); i++;
GET_COUNTER();
/* rx bytes */
CONFD_SET_UINT64(&val[i], counter); i++;
GET_COUNTER();
/* rx packets */
CONFD_SET_UINT64(&val[i], counter); i++;
GET_COUNTER();
/* rx errs */
CONFD_SET_UINT32(&val[i], counter); i++;
GET_COUNTER();
/* rx drop */
CONFD_SET_UINT32(&val[i], counter); i++;
/* skip remaining rx counters */
GET_COUNTER(); GET_COUNTER(); GET_COUNTER(); GET_COUNTER();
CONFD_SET_XMLTAG(&val[i], if_transmit, if__ns); i++;
GET_COUNTER();
/* tx bytes */
CONFD_SET_UINT64(&val[i], counter); i++;
GET_COUNTER();
/* tx packets */
CONFD_SET_UINT64(&val[i], counter); i++;
GET_COUNTER();
/* tx errs */
CONFD_SET_UINT32(&val[i], counter); i++;
GET_COUNTER();
/* tx drop */
CONFD_SET_UINT32(&val[i], counter); i++;
GET_COUNTER();
/* skip */
GET_COUNTER();
/* tx colls */
CONFD_SET_UINT32(&val[i], counter); i++;
ret = cdb_set_object(sock, val, i,
"/interfaces/interface{%s}/status", ifname);
if (ret == CONFD_ERR && confd_errno == CONFD_ERR_BADPATH)
/* assume interface doesn't exist in config */
ret = CONFD_OK;
}
fclose(proc);
cdb_end_session(sock);
return ret;
}

We typically call this function at regular intervals.

6.9. Delayed Replies
If the data source is communicated with through some means of IPC it may be inconvenient to hang in the
callback functions and wait for the reply from the data source. The solution to this problem is to return a
special return value from the callback and then later explicitly send the response once it is available.
All the transaction callbacks as well as all the data callbacks can optionally return the value
CONFD_DELAYED_RESPONSE. This means that the callback returns, and we typically end up in our
main poll loop again. Once the reply returns it is then up to the application to send the reply back to ConfD.
The libconfd library contains a number of routines that can be invoked to convey a delayed
response. The callbacks are divided in two groups. The first group is the one where the actual return

100

Operational Data

value from the callback is the value that is sent to ConfD as a response. A good example is the
the transaction init() callback or the the data callback set_elem(). In both these case if the
callback returns CONFD_OK a positive ack is sent back to ConfD by the library. If we instead return
CONFD_DELAYED_RESPONSE the application must - once the reply is available - use either of the
functions confd_delayed_reply_ok() or confd_delayed_reply_error() to explicitly
send the reply. If no reply is sent within 120 seconds (configurable through confd.conf) the data
provider is considered dead by ConfD and ConfD will close all sockets to the data provider.
Another group of callbacks are the callbacks that require the application to explicitly send a
reply back to ConfD before returning. A good example is the data callback get_elem(). The
application must explicitly call confd_data_reply_value() before returning - unless the
CONFD_DELAYED_RESPONSE value is returned. If so, it is up to the application to later, when the
response value is available, explicitly call the confd_data_reply_value() function to send back
the return value.

6.10. Caching Operational Data
For operational data handled by an external data provider (i.e., using tailf:callpoint), the values
of elements may be kept for a certain time in a cache in ConfD. If such an element is accessed, its value
will be taken from the cache, and the data provider not called.
The cache is enabled, and the default time to keep values in the cache configured, with the element /
confdConfig/opcache in the confd.conf file, for example:

true
5


By default, the cache is disabled. The timeout value is given in seconds, it does not have a default. If confd
--reload is done, the cache will use the new timeout value. If the cache is disabled, the stored values are
cleared.
To indicate that the elements handled by a callpoint are to be saved in the cache, use the tailf:cache
statement:
leaf packetCounter {
type uint64;
config false;
tailf:callpoint a1 {
tailf:cache true;
}
}

It is also possible to override the cache timeout specified in confd.conf by using the tailf:timeout
substatement with tailf:cache in the data model.
leaf packetCounter {
type uint64;
config false;
tailf:callpoint a1 {
tailf:cache true {
tailf:timeout 7;
}
}

101

Operational Data

}

The timeout specified this way will be used for the node with the tailf:timeout statement and any
descendants of that node, unless another tailf:cache statement is used on a descendant node. Using
tailf:cache without a tailf:timeout substatement will cause the timeout to revert to the one
specified in confd.conf.
The results of get_next() and find_next() operations can not be cached in general, since the next
value returned by the data provider does not necessarily identify a specific list entry (e.g. it could be a
fixed pointer to a data structure holding the "next entry" information). However in the special case that the
data provider returns -1 for next the result can be cached, since retrieval of the next entry will then use
a find_next operation with the complete set of keys from the previous entry. See the confd_lib_dp(3)
manual page for further details.
The cache can be cleared, partially or completely, by means of the maapi_clear_opcache() function
- see the confd_lib_maapi(3) manual page.

6.11. Operational data lists without keys
It is possible to define lists for operational data without any keys in the YANG data model, e.g.:
list memory-pool {
config false;
tailf:callpoint memstats;
leaf buffer-size {
type uint32;
}
leaf number-of-buffers {
type uint32;
}
}

To support this without having completely separate APIs, we use a "pseudo" key in the ConfD APIs for
this type of list. This key is not part of the data model, and completely hidden in the northbound agent
interfaces, but is used with e.g. the get_next() and get_elem() callbacks as if it were a normal key.
This "pseudo" key is always a single signed 64-bit integer, i.e. the confd_value_t type is C_INT64. The
values can be chosen arbitrarily by the application, as long as a key value returned by get_next() can
be used to get the data for the corresponding list entry with get_elem() or get_object() as usual.
It could e.g. be an index into an array that holds the data, or even a memory address in integer form.
There are some issues that need to be considered though:
• In some cases ConfD will do an "existence test" for a list entry. For "normal" lists, this is done by
requesting the first key leaf via get_elem(), but since there are no key leafs, this can not be done.
Instead ConfD will use the exists_optional() callback for this test. I.e. a data provider that has
this type of list must implement this callback, and handle a request where the keypath identifies a list
entry.
• In the response to the get_next_object() callback, the data provider is expected to provide
the key values along with the other leafs in an array that is populated according to the data
model. This must be done also for this type of list, even though the key isn't actually in
the data model. The "pseudo" key must always be the first element in the array, and for the
confd_data_reply_next_object_tag_value_array() reply function, the tag value 0
should be used. Note that the key should not be included in the response to the get_object()
callback.

102

Operational Data

• The same approach is used when we store operational data in CDB - the path used in the write (and
read) functions in the CDB API must include the "pseudo" integer key. If multiple list entries are to
be written with a single call to cdb_set_values(), which takes a tagged value array, the key for
each entry must be included in the array with a tag value of 0, in the same way as described above. This
applies also to reading multiple entries with a single call to cdb_get_values().

103

Chapter 7. The external database API
7.1. Introduction to external data
In the previous chapter we showed how to use ConfD with read-only operational data. In this chapter we
will use the same APIs from libconfd.so to implement externally stored configuration data. This is
the opposite of CDB, if CDB is used to store the configuration data, this section can be skipped.
We show how ConfD can use an external database as data source. The external data base can either be a
full-fledged real data base or something as simple as a text file.
The configuration of the network device is modeled by a YANG module. It describes the data model of
the device and ConfD needs to populate the XML data tree with actual data.
If the ConfD built-in XML database (CDB) is used to hold all configuration data, ConfD will automatically
read and write into that database. If, on the other hand, the actual configuration data is kept outside of
ConfD we need user supplied code to provide ConfD with the actual data of the configuration.

7.2. Scenario - The database is a file
Many standard UNIX applications read their configuration from a static file. If we want to integrate such
an application into our network device, it may not be feasible to rewrite the application so that it reads
its configuration from the device configuration database. In general we want to change the code of the
application as little as possible.
Examples of such applications are abundant. In general this applies to all open source applications
generally found on UNIX machines.
In order to integrate such an application into ConfD we must first write a YANG module which models
the part of the application (the part of the application's configuration file) which we wish to be able
to configure. Following that we must write C code which can read, parse, manipulate and write the
configuration file in question and finally we must connect that C code to ConfD.
We did precisely this exercise in Chapter 5, CDB - The ConfD XML Database, however the solution from
that chapter had the actual configuration data in CDB, and the configuration file was generated. Thus,
if the file was edited or otherwise changed externally, those changes would be overwritten the next time
we regenerated the file. In this chapter we will show how to use the actual file as a database. I.e. no
configuration data is ever kept inside ConfD, the data resides outside ConfD.

7.3. Callpoints and callbacks
Similar to how we managed operational data, we need to define a data model and annotate the model with
a callpoint.
Assume that we wish to model a set of 'server' structures as in the following YANG module:

Example 7.1. A list of server structures
module smp {
namespace "http://tail-f.com/ns/example/smp";
prefix smp;

104

The external database API

import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
/* A set of server structures */
container servers {
tailf:callpoint simplecp;
list server {
key name;
max-elements 64;
leaf name {
type string;
}
leaf ip {
type inet:ipv4-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
}
}
}

The callpoint called simplecp instructs ConfD that whenever it needs to populate the XML tree below
simplecp, it must invoke callbacks in an external program which has registered itself with the name
simplecp. The external programs use the API in libconfd.so to register themselves under different
callpoints.

7.4. Data Callbacks
When we implemented the operational data callbacks we had to implement a set of callbacks
for each callpoint. With external data we must do the same, but some additional callbacks
must also be implemented. The data callbacks get_next(), get_elem(), get_object(),
get_next_object(), find_next(), find_next_object(), num_instances(), and
finally exists_optional() work precisely the same for external data as they do for operational data.
Those callbacks are thus described in the previous chapter.
Additionally the following data callback functions are required for external data:
• create() - This callback creates a new list entry. In the case of the smp.yang module above, this
function needs to create a new empty "server" entry. Once the entry is created, it will be populated with
values through a series of calls to set_elem().
• remove() - This callback needs to remove an entire list entry and all its subelements.
• set_elem() - This callback sets the value of a leaf.

7.5. User sessions and ConfD Transactions
Again, similar to the chapter on operational data user sessions are created when a user logs in, and new
transactions are created when an agent initiates an activity.

105

The external database API

If we deal with operational data, the different phases are not interesting, thus then we only had to implement
the init() and a finish() callback. This section describes the states of a ConfD transaction and also
which user callbacks that need to be implemented in order to participate in the transaction.
In a device where ConfD is used to manage the configuration data there can be multiple sources of data.
To use ConfD terminology: there can be several different daemons that connect to ConfD under different
callpoints. Some callpoints may also be served by CDB.
Furthermore, a set of write operations may involve several of these daemons as well as CDB. In order
to ensure that all participants perform the operations, ConfD orchestrates a two-phase commit protocol
towards the different participants. Each NETCONF operation, such as edit-config or each call to
commit in the CLI will be clumped into a ConfD transaction. If we store our data outside of ConfD - as
will be described in this chapter - we must implement a number of callback functions in order to participate
in the various states of the transaction.
An individual daemon may (or may not) implement the callbacks for the two-phase commit protocol. If
there is only one daemon and CDB is not used at all, the two-phase commit protocol may be skipped. The
reason for this is that when there is only one participant, the two-phase commit protocol is irrelevant.
Each NETCONF operation, i.e. each edit-config and so forth, will execute as one transaction. Thus
transactions originating from NETCONF will be fairly short-lived entities whereas transactions originating
from the CLI or the Web UI will be longer.
A daemon that wishes to participate in the two-phase commit transaction must implement a number of
callback functions.
• init() - As for operational data, from the daemon's point of view the init() callback is invoked
when a transaction starts, but ConfD delays the actual invocation as an optimization. For a daemon
providing configuration data, init() is invoked just before the first data-reading callback, or just
before the trans_lock() callback (see below), whichever comes first. When a transaction has
started, it is in a state we refer to as READ. ConfD will, while the transaction is in the READ state, execute
a series of read operations towards (possibly) different callpoints in the daemon.
Any write operations performed by the management station are accumulated by ConfD and the daemon
doesn't see them while in the READ state.
• trans_lock() - This callback gets invoked by ConfD at the end of the transaction. ConfD has
accumulated a number of write operations and will now initiate the final write phases. Once the
trans_lock() callback has returned, the transaction is in the VALIDATE state. In the VALIDATE
state, ConfD will (possibly) execute a number of read operations in order to validate the new
configuration. Following the read operations for validations comes the invocation of one of the
write_start() or trans_unlock() callbacks.
• trans_unlock() - This callback gets invoked by ConfD if the validation failed or if the validation
was done separate from the commit (e.g. by giving a validate command in the CLI). Depending on where
the transaction originated, the behavior after a call to trans_unlock() differs. If the transaction
originated from the CLI, the CLI reports to the user that the configuration is invalid and the transaction
remains in the READ state whereas if the transaction originated from a NETCONF client, the NETCONF
operation fails and a NETCONF rpc error is reported to the NETCONF client/manager.
• write_start() - If the validation succeeded, the write_start() callback will be called and
the transaction enters the WRITE state. While in WRITE state, a number of calls to the write callbacks
set_elem(), create() and remove() will be performed.
If the underlying database supports real atomic transactions, this is a good place to start such a
transaction.

106

The external database API

The application should not modify the real running data here. If, later, the abort() callback is called,
all write operations performed in this state must be undone.
• prepare() - Once all write operations are executed, the prepare() callback is executed. This
callback ensures that all participants have succeeded in writing all elements. The purpose of the callback
is merely to indicate to ConfD that the daemon is ok, and has not yet encountered any errors.
• abort() - If any of the participants return an error or fail to reply in the prepare() callback,
the remaining participants all get invoked in the abort() callback. All data written so far in this
transaction should be disposed of.
• commit() - If all participants successfully replied in their respective prepare() callbacks, all
participants get invoked in their respective commit() callbacks. This is the place to make all data
written by the write callbacks in WRITE state permanent.
• finish() - And finally, the finish() callback gets invoked at the end. This is a good place to
deallocate any local resources for the transaction.
The finish() callback can be called from several different states.
The following picture illustrates the conceptual state machine a ConfD transaction goes through.

ConfD transaction state machine
All callbacks except the init() callback are optional. If a callback is not implemented, it is the same
as a succeeding empty implementation such as:
int mycallback(struct confd_trans_ctx *tctx)
{

107

The external database API

return CONFD_OK;
}

In the following examples, we will initially not use these transactions at all. We will implement the
init() callback only and let the other transaction callbacks be NULL.

7.6. External configuration data
In this section we provide a commented example which manages actual configuration data. The idea is that
ConfD runs the NETCONF agent and is entirely responsible for the candidate configuration and possibly
runs the CLI and the Web UI. The application is responsible for maintaining and storing the configuration
data.
An actual running version of this example can be found in the examples directory of a ConfD release under
user_guide_examples/simple_no_trans.
The example system stores "servers" with name, ip, and port on a file. Our YANG module will be very
simple; we have:

Example 7.2. The smp.yang module
module smp {
namespace "http://tail-f.com/ns/example/smp";
prefix smp;
import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
/* A set of server structures */
container servers {
tailf:callpoint simplecp;
list server {
key name;
max-elements 64;
leaf name {
type string;
}
leaf ip {
type inet:ipv4-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
}
}
}

To implement this we first need a small database. We choose to use a simple array of "server" structures,
as in:
struct server {
char name[256];

108

The external database API

struct in_addr ip;
unsigned int port;
};
static struct server running_db[64];
static int num_servers = 0;

To create a new "server" in the database we add a new server structure to the array, as in:
static struct server *add_server(char *name)
{
int i, j;
for (i=0; i < num_servers; i++) {
if (strcmp(running_db[i].name, name) > 0) {
/* found the position to add at, now shuffle the */
/* remaining elems in the array one step */
for (j = num_servers; j > i; j--) {
running_db[j] = running_db[j-1];
}
break;
}
}
num_servers++;
memset(&running_db[i], 0, sizeof(struct server));
strcpy(running_db[i].name, name);
return &running_db[i];
}
static struct server *new_server(char *name, char *ip, char *port)
{
struct server *sp = add_server(name);
sp->ip.s_addr = inet_addr(ip);
sp->port = atoi(port);
return sp;
}

We keep the array ordered according to the key (server name), since ConfD expects us to return entries
in that order when traversing the list.
Note that at first glance this code looks like we may write off the end of the running_db array. But
this is not the case, since the server list in the data model is defined with max-elements 64;. This
means that ConfD will guarantee that there are never more than 64 servers.
To search the database for a specific server we have:
/* Find a specific server */
static struct server *find_server(confd_value_t *v)
{
int i;
for (i=0; i < num_servers; i++) {
if (confd_svcmp(running_db[i].name, v) == 0)
return &running_db[i];
}
return NULL;
}

Our find_server() function utilizes a strcmp()-like function from libconfd.so - the function
confd_svcmp() compares a string char* value to a confd_value_t value. The type of the confd_value_t
must obviously be either a string or a buffer.

109

The external database API

The initialization code is very similar to the ARP example in the chapter on operational data, with the
exception that we must also here register functions to write new data. We need to register callbacks to
set_elem() which set the value of a leaf element such as /servers/server{www}/ip. We also
need to register callback functions that can create a new "server" entry and delete old "server" entries.
Thus we initialize our data callback structure struct confd_data_cbs as:
data.get_elem
data.get_next
data.set_elem
data.create
data.remove

=
=
=
=
=

get_elem;
get_next;
set_elem;
create;
doremove;

The get_elem() and get_next() callbacks can be implemented in a manner similar to how we
implemented the corresponding callbacks for the ARP example. For example:

Example 7.3. get_next() callback for smp.yang
static int get_next(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath,
long next)
{
confd_value_t v;
if (next == -1) { /* Get first key */
if (num_servers == 0) { /* Db is empty */
confd_data_reply_next_key(tctx, NULL, -1, -1);
return CONFD_OK;
}
CONFD_SET_STR(&v, running_db[0].name);
confd_data_reply_next_key(tctx, &v, 1, 1);
return CONFD_OK;
}
if (next == num_servers) { /* Last elem */
confd_data_reply_next_key(tctx, NULL, -1, -1);
return CONFD_OK;
}
CONFD_SET_STR(&v, running_db[next].name);
confd_data_reply_next_key(tctx, &v, 1, next+1);
return CONFD_OK;
}

The create callback is easy. The keypath passed to the create() callback will have the new key (last in
the string) as first element (in the array). Recall that the keypaths are passed in reversed order. For example
when ConfD wants to create a new server entry, named to for example "smtp", the keypath will look like
/servers/server{smtp}.
The data model can optionally specify default values. In smp.yang we didn't use that feature. For example
the "port" leaf was specified as:
leaf port {
type inet:port-number;
mandatory true;
}

and not as
leaf port {

110

The external database API

type inet:port-number;
default 0;
}

Our C code needs to be able to create list entries in the database without any of the actual values of the
leafs given. All keys will be given but none of the actual values of the other leafs (except for the key
leafs). ConfD will set all the missing values using the set_elem() callback. Our create() callback
looks like:

Example 7.4. create() callback for smp.yang
static int create(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
confd_value_t *key = &keypath->v[0][0];
add_server((char *)CONFD_GET_BUFPTR(key));
return CONFD_OK;
}

In a similar manner, the remove() callback deletes a server entry.

Example 7.5. remove() callback for smp.yang
static int doremove(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
int i, j;
confd_value_t *key = &keypath->v[0][0];
for (i=0; i < num_servers; i++) {
if (confd_svcmp(running_db[i].name, key) == 0) {
/* found the elem to remove, now shift the */
/* remaining elems in the array one step */
for (j=i+1; j < num_servers; j++) {
running_db[j-1] = running_db[j];
}
num_servers--;
return CONFD_OK;
}
}
return CONFD_OK;
}

Finally here is the set_elem() callback which is responsible for setting a leaf value. The code is:

Example 7.6. set_elem() callback for smp.yang
static int set_elem(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath,
confd_value_t *newval)
{
confd_value_t *tag = &(keypath->v[0][0]);
struct server* s = find_server(&(keypath->v[1][0]));
if (s == NULL) {
confd_trans_seterr(tctx, "no such server found");
return CONFD_ERR;
}

111

The external database API

switch (CONFD_GET_XMLTAG(tag)) {
case smp_ip:
s->ip = CONFD_GET_IPV4(newval);
break;
case smp_port:
s->port = CONFD_GET_INT32(newval);
break;
default:
return CONFD_ERR;
}
return CONFD_OK;
}

Note that there is no switch clause for smp_name - ConfD will never change key values by invoking
set_elem() for key leafs. Changing keys can only be done by a combination of remove() and
create() invocations, followed by set_elem() invocations for the non-leaf keys in the created list
entry.

7.7. External configuration data with
transactions
In this section we introduce and use the transaction callbacks.
An actual running version of this example can be found in the examples directory of a ConfD release under
user_guide_examples/simple_trans.
An application is invoked in trans_lock() when a transaction is committed or when a transaction is
validated (e.g. by doing validate in the CLI), and the transaction enters the VALIDATE state.
When the application is invoked in the trans_lock() callback, the following is guaranteed.
• A sequence of callbacks will be invoked without delays. ConfD has accumulated a number of write()
operations and will execute them in a sequence without delays.
• No callbacks to any other transactions towards the same data store will be executed between the
invocation of trans_lock() and the invocation of finish() (or trans_unlock()). Thus all
transactions towards a given data store are serialized once they reach the VALIDATE state.
After validation, either trans_unlock() or write_start() is invoked. trans_unlock() is
called when the transaction is validated only, and write_start() is called when the validation was
done as the first part of the commit, and validation succeeded.
If the underlying database is a real database with real support for transactions, it is a very good idea to start
such a native transaction in the call to write_start(). If that is not the case the libconfd.so library
provides support which makes it possible to accumulate the write operations without actually writing them.
In this example we save the database to a file for persistence

Example 7.7. save() utility function
static int save(char *filename) {
FILE *fp;
int i;

112

The external database API

if ((fp = fopen(filename, "w")) == NULL)
return CONFD_ERR;
for (i=0; i < num_servers; i++) {
fprintf(fp, "%s %s %d\n",
running_db[i].name,
inet_ntoa(running_db[i].ip),
running_db[i].port);
}
fclose(fp);
return CONFD_OK;
}

We instantiate all the transaction callbacks and do the appropriate thing in each callback. Since the
database is just a simple array, the variable running_db, we choose to let the library libconfd.so
accumulate the individual write operations by returning CONFD_ACCUMULATE from the write callbacks
set_elem(), create() and remove(). The data will be copied into data structures in the library.
The purpose of doing this is that we do not want to explicitly write into our local data structures in the write
routines - rather we wish to delay this and perform the actual write operations in the prepare() callback.

Example 7.8. write callbacks using accumulate
static int set_elem(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath,
confd_value_t *newval)
{
return CONFD_ACCUMULATE;
}
static int create(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
return CONFD_ACCUMULATE;
}
static int doremove(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
return CONFD_ACCUMULATE;
}

We are thus not doing anything at all in the write callbacks, except returning the value
CONFD_ACCUMULATE. Note that this will store a complete copy of the keypath and also of the new value
if the operation is set_elem().
All the operations will be copied and kept in a linked list in the transaction context (struct confd_trans_ctx).
In the PREPARED state we will loop through all the operations and perform them.
Remember the reason for implementing the two-phase commit protocol. There may be multiple daemons
connected to ConfD and a series of write operations, i.e a transaction may span several daemons. ConfD
ensures that e.g. a commit from the CLI is either written in all of the connected daemons or none - thus
ensuring a consistent database.
Recall the picture depicting the state transitions:

113

The external database API

ConfD transaction state machine
The most complicated callback is prepare():

Example 7.9. prepare() callback using the accumulated write ops
static int t_prepare(struct confd_trans_ctx *tctx)
{
struct server *s;
struct confd_tr_item *item = tctx->accumulated;
while (item) {
confd_hkeypath_t *keypath = item->hkp;
confd_value_t *leaf = &(keypath->v[0][0]);
switch(item->op) {
case C_SET_ELEM:
s = find_server(&(keypath->v[1][0]));
if (s == NULL)
break;
switch (CONFD_GET_XMLTAG(leaf)) {
case smp_ip:
s->ip = CONFD_GET_IPV4(item->val);
break;
case smp_port:
s->port = CONFD_GET_INT32(item->val);
break;
}
break;
case C_CREATE:

114

The external database API

add_server((char *)CONFD_GET_BUFPTR(leaf));
break;
case C_REMOVE:
remove_server(leaf);
break;
default:
return CONFD_ERR;
}
item = item->next;
}
return save("running.prep");
}

The above code loops through all the struct confd_tr_item structs accumulated by the library in the
accumulated field for the transaction context.
The accumulated write structs are defined as:
enum confd_tr_op {
C_SET_ELEM = 1,
C_CREATE= 2,
C_REMOVE = 3,
C_SET_CASE = 4,
C_SET_ATTR = 5,
C_MOVE_AFTER = 6
};
struct confd_tr_item {
char *callpoint;
enum confd_tr_op op;
confd_hkeypath_t *hkp;
confd_value_t *val;
confd_value_t *choice; /* only for set_case */
u_int32_t attr;
/* only for set_attr */
struct confd_tr_item *next;
};

If we had a real native database with real transaction support, we wouldn't have used the accumulation
feature of the library at all - rather we would have started a native transaction in the write_start()
callback.
Our example database is just an array and a file; thus we use the accumulation feature of the library.
In the prepare() callback we finally save the database to a file called running.prep - thus preparing
to commit the changes we have made.
The corresponding abort() and commit() callbacks are easy:

Example 7.10. commit() and abort()
static int t_commit(struct confd_trans_ctx *tctx)
{
if (rename("running.prep", "running.DB") == 0)
return CONFD_OK;
else
return CONFD_ERR;
}
static int t_abort(struct confd_trans_ctx *tctx)

115

The external database API

{
restore("running.DB");
unlink("running.prep");
return CONFD_OK;
}

The restore() reads a file and initializes the database (our array) from that file:

Example 7.11. Code to restore our array from a file
static int restore(char *filename)
{
char buf[BUFSIZ];
FILE *fp;
if ((fp = fopen(filename, "r")) == NULL)
return CONFD_ERR;
num_servers = 0;
while (fgets(&buf[0], BUFSIZ, fp) != NULL) {
char *name, *ip, *port;
if ((name = strtok(buf, " \t\r\n")) != NULL &&
((ip = strtok(NULL, " \t\r\n")) != NULL) &&
((port = strtok(NULL, " \t\r\n")) != NULL)) {
printf("Loaded %s\n", name);
new_server(name, ip, port);
}
}
return CONFD_OK;
}

7.8. Writable operational data
Writable operational data is indicated in the YANG model as config false marked with
tailf:writable true. This is typically used when an SNMP MIB has data that models an operation,
like "reboot". For other interfaces than SNMP, such an operation should be modeled as an rpc or action.
Writable operational data must be implemented by callback functions, just like external configuration data,
as described in Section 7.7, “External configuration data with transactions”. When a transaction is started
for operational data, the dbname field in struct confd_trans_ctx is CONFD_OPERATIONAL.

7.9. Supporting candidate commit
The NETCONF protocol has as one of its major features the concept of candidate commit with a timeout.
The manager manipulates the candidate configuration and finally commits the candidate. This means that
the candidate configuration is copied into the running data base and thus is active.
If the commit operation is accompanied by a timeout then the semantics is that if the application has not
received a confirming commit before the timeout, the previous running configuration should be copied
back into running. The idea here is that if a configuration is somehow bad, an automatic rollback will
occur.
There are several different usage scenarios whereby this feature is supported with ConfD.
• The by far easiest case is when the database is kept in the ConfD built-in XML database, CDB. When
that is the case, candidate commit is supported directly by ConfD natively.

116

The external database API

• The next case is when the candidate configuration is managed by ConfD but the running
configuration is kept outside ConfD. This is described here. The application needs to register three
checkpoint callbacks in the database callback struct confd_db_cbs by means of the API call
confd_register_db_cb().
• The final case is when both the running and the candidate configuration are kept entirely outside
of ConfD. Remember the ConfD transactions that get executed. When a new transaction is started, one
of the fields in the transaction context, the dbname field indicates which database the transaction is
started for.
If ConfD owns the candidate, no transactions will ever be created towards the candidate. If
the application owns both running and the candidate (as configured in confd.conf) then
transaction may be directed towards either running or candidate.
In the case where the candidate is owned by the application, the application needs to register six
candidate callbacks in the database callback struct struct confd_db_cbs by means of the API call
confd_register_db_cb(). This mode of operations only make sense if the external database can
truly support the candidate callbacks. If that is not the case it i better to let ConfD manage the candidate.
In this section we provide an example where ConfD owns the candidate datastore. The application needs
to register the following callbacks.
1. add_checkpoint_running() - This callback must create a checkpoint of the current running
configuration and store it in non-volatile memory. When the system restarts, it is the responsibility of
the external application to check if there is a checkpoint available, and use the checkpoint instead of
running.
2. del_checkpoint_running() - This function must delete a checkpoint created by
add_checkpoint_running(). It is called by ConfD when a confirming commit is received.
3. activate_checkpoint_running() - This function should rollback running to the checkpoint
created by add_checkpoint_running(). It is called by ConfD when the timer expires or if the
user session expires. There can be at most one checkpoint live at a time.
Using our previous save() and restore() functions the implementation of the checkpoint callbacks
becomes very simple.

Example 7.12. checkpoint db callbacks
add_checkpoint_running(struct confd_db_ctx *db)
{
return save("running.checkpoint");
}
del_checkpoint_running(struct confd_db_ctx *db)
{
unlink("running.checkpoint");
return CONFD_OK;
}
activate_checkpoint_running(struct confd_db_ctx *db)
{
return restore("running.checkpoint");
}

Two things remain to be done. First we need to register the checkpoint callbacks. Second we need to look
for the existence of a saved checkpoint when we initialize our database and if it exists, running should
be initialized from the checkpoint instead. Thus:

117

The external database API

/* global variable */
static struct confd_db_cbs dbcbs;
...
int main()
{
...
if ((restore("running.checkpoint")) != CONFD_OK)
restore("running.DB");

dbcbs.add_checkpoint_running = add_checkpoint_running;
dbcbs.del_checkpoint_running = del_checkpoint_running;
dbcbs.activate_checkpoint_running = activate_checkpoint_running;
/* register the callbacks */
confd_register_db_cb(dctx, &dbcbs);
confd_register_done(dctx);

If the underlying database is a real database we would install database checkpoints instead of copying
entire files back and forth.
If we choose to implement the checkpoint callbacks as above, we must obviously also configure ConfD
accordingly. The relevant sections in confd.conf from the datastores section are:

true
confd


And from the NETCONF section:


true


true



Finally, if we implement the database outside ConfD we may optionally choose to implement the lock()
and unlock() callbacks. This is only interesting if there exists additional locking mechanisms towards
the database - such as an external CLI which can lock the database, of if the external database owns the
candidate.

7.10. Discussion - CDB versus external DB
In this section we discuss some of the requirements that an external database must be able to fulfill in order
for ConfD to work properly. The reasons for choosing an external database as opposed to CDB may vary
between projects. Some projects already have a database and the managed object code is already tightly
coupled to that database. Other projects may feel that the underlying database must have characteristics
that CDB doesn't have. It is certainly the case that CDB is not the best choice for, for example distributed

118

The external database API

replication of large amounts of state data. CDB is not a check-pointing database for application state
replication.
The first and most important requirement ConfD has on an external database is that it can execute
transactions. The transaction manager inside ConfD will collect all data for a transaction and once the
data has been validated, it will send the data as a series of write operations to the data provider. It is
the responsibility of the database to execute this series of write operations atomically. Either they all get
written or none. External databases that do not support transactions can still be used of course, but that
then comes with the possibility of getting a corrupt configuration. Corruption will occur if:
1. Another data provider rejects the transaction - in this case ConfD will tell all data providers to abort. If
there are no other data providers than the external database - this cannot happen.
2. ConfD dies while sending the write operations to the data provider, alternatively the network
connectivity between ConfD and the data provider breaks. If this happens, the data provider never
gets the whole transaction. One way of partially addressing this problem may be to make use of
CONFD_ACCUMULATE feature whereby all writes are accumulated inside the library. That way the
data provider at least can be certain that it has the entire transaction prior to starting its own write session.
Furthermore, CDB has two important features, schema upgrade and subscriptions. An external database
must at least address this functionality.
Schema upgrade. When the YANG data model files are changed, CDB has the old schema - and its
associated data - stored. On upgrade, CDB transforms all the old data so that it adheres to the new schema.
If CDB is not used, the equivalent functionality must be performed by the external database.
Subscriptions - when the configuration is changed - the applications, the consumers of configuration data,
must somehow be notified of the configuration changes. If CDB is not used, this is now the task of the
external database.
Finally, if an external database is used, we must provide a mapping in the code of the data provider
between ConfD keypaths and values to entries in the external database. For example, if we use a simple
key/value database it's possible to write general code that works for all possible keypaths. The key is
a confd_hkeypath_t and the value is obviously a confd_value_t. The only problem is how to handle
create() and delete() operations for a key/value database. In the case of a delete operation, all
children must also be deleted. It is easy to find the children since the schema is loaded in a data provider
(through confd_load_schemas()) and a key/value data provider would then have to follow the
schema, and delete all children.

119

Chapter 8. Configuration Meta-Data
8.1. Introduction to Configuration Meta-Data
In ConfD, meta-data can be associated with configuration data nodes. The meta-data is stored as
attributes on data nodes in the configuration datastore. Having meta-data is optional, and requires
support from the datastore implementation. CDB (see Chapter 5, CDB - The ConfD XML Database fully
supports meta-data attributes, but if an external data provider (see Chapter 7, The external database API
is used for configuration data, it needs to explicitly support meta-data attributes.
There are three meta-data attributes in ConfD, annotation, tag, and inactive. Each of these is
discussed in the following sections.
To enable meta-data attributes, /confdConfig/enableAttributes in confd.conf (see
confd.conf(5)) must be set to true.

8.2. Meta-Data: annotation
Any configuration data node can have at most one annotation attribute. An annotation is an arbitrary
string which acts a comment for the node.
In the CLI, an annotation is set with the annotate command, and displayed as a comment. See
Section 16.15, “Annotations and tags” for details.
admin@host% annotate interface eth0 "mgmt interface"
admin@host% show interface
/* mgmt interface */
interface eth0 {
...
}

In NETCONF, an annotation is created and display as an XML attribute, see Section 15.15, “Meta-data
in Attributes” for details.
Annotations are not visible in the CDB API for CDB subscribers.

8.3. Meta-Data: tag
Any configuration data node can have a set of tags associated with it. Tags are set by the user for data
organization and filtering purposes.
In the CLI, tags are administered with the tag command, and displayed as a comment with special syntax.
It is also possible to filter the configuration based on how it is tagged. See Section 16.15, “Annotations
and tags” for details.
In NETCONF, a tag is created and display as an XML attribute, see Section 15.15, “Meta-data in
Attributes” for details. Standard XPath filtering can be used to filter the configuration based on how it
is tagged.
Tags are not visible in the CDB API for CDB subscribers.

120

Configuration Meta-Data

8.4. Meta-Data: inactive
Any existing, deletable data node can be marked as inactive. This has the same effect as deleting the
node, except that it is still kept in the configuration data store, marked as being inactive.
To enable support for inactive nodes, /confdConfig/enableInactive in confd.conf (see
confd.conf(5)) must be set to true. All configuration data providers must support the inactive
attribute.
In the CLI, the command deactivate makes a node inactive, and the command activate activates an inactive
node. See See Section 16.16, “Activate and Deactivate” for details.
In NETCONF, a separate capability is used by the server to announce that it supports inactive nodes. Clients
must use special parameters to tell the server that they understand the inactive attribute. See Section 15.12,
“Inactive Capability” for details.
Inactive nodes are not visible in the CDB API for CDB subscribers. If a node is inactivated, a CDB
subscriber will see the node as being deleted, and when it is activated, the CDB subscriber will see it as
being created.
Inactive nodes are not visible in validation code (see Chapter 9, Semantic validation). Validation
constraints defined in the data model (e.g. max-elements or must) do not take inactive nodes into
account.
Since data providers must support the inactive attribute, all hooks and transforms (see Chapter 10,
Transformations, Hooks, Hidden Data and Symlinks) will see the inactive nodes being marked as inactive,
and must be explicitly coded to handle this attribute.
admin@host% deactivate interface eth0
admin@host% show interface
inactive: interface eth-0 {
...
}

121

Chapter 9. Semantic validation
9.1. Why Do We Need to Validate
ConfD stores device configuration data. Some device configuration data is truly critical for the correct
operations of the device. Misconfiguring a network device may lead to a situation where the device is no
longer connected to the network. Before committing configuration data it is crucial to ensure that the new
configuration is correct.
Another benefit with a guaranteed correct configuration, is that application software which reads the
configuration data need not check the validity of the configuration.
ConfD has support for several different levels of validation. We have:
• Syntactic validation - this means that the configuration data - viewed as an XML document - must
adhere to the YANG model.
• Integrity constraints. Certain configuration leaves may only have values within specified ranges.
• YANG must statements use XPath expressions that can be used to constrain values. This is a very
powerful mechanism whereby it's possible to instruct ConfD to compute an XPath expression whenever
a configuration change is attempted. This makes it possible to have value constraints that depend on
other parts of the configuration.
• Explicit validation logic where user code gets to read and analyze the configuration prior to commit.

9.2. Syntactic Validation in YANG models
A YANG model is a schema. It has a number of constructs the define the structure of the model as a whole
as well as type constrains on individual leaves.
Structure enforcing statements include constructs like container presence statements, leaf mandatory
statements, leaf-list min and max elements statements and list min and max elements statements.
Each leaf has a either a built-in primitive type, e.g. integer, string, boolean etc, or a derived type e.g. a
union, enumeration, boundary restriction, or regular expression pattern
The ConfD CLI and Web UI use this information to guide the operator what is possible to configure.

9.3. Integrity Constraints in YANG Models
Before going for semantic validation we should also make sure that our need for validation can not be
satisfied by any of the integrity constraint constructs available in the YANG model modeling language:
min-elements and maxelements

Specifies how many instances may exist in the configuration data
store. Both YANG list statements and leaf-list statements can be
constrained by a min and/or a max.

key

Specifies that the leaf is used as a key for a multi-instance object.
An object can have multiple keys.

unique

Specifies that the leaf's value must be unique across all instances.

122

Semantic validation

leafref

The leafref type is used to reference a particular leaf instance in the
data tree. Its value is constrained to be the same as the value of an
existing leaf.

Read more about integrity constraints in the Chapter 3, The YANG Data Modeling Language chapter as
well in http://www.ietf.org/rfc/rfc6020.txt.

9.4. The YANG must Statement
Using XPath expressions it is possible to express constraints on values in virtually any way. XPath is
complicated and requires a bit of work to learn. It is well worth the effort though, since writing declarative
constraints on the data model - in the data model - is better than writing C code that executes outside of
ConfD.

9.5. Validation Logic
Semantic validation in ConfD extends the validation functionality to allow for verification of constraints
that can not be expressed by the above constructs. It is important to realize that the basic concept is the
same, though. The task of semantic validation is to make sure that the new configuration satisfies some
set of logical constraints before it is allowed to be committed.
With a command centric view of configuration, validation may be thought of as checking the validity
of operator actions vis-a-vis existing configuration. This tends to lead to complex and error prone code,
since there will often be a large number of combinations of actions and configuration values that need
to be checked, and some "corner cases" can easily be overlooked. Furthermore covering multiple user
interfaces, such as NETCONF and Web UI in addition to CLI, with the same validation code will be an
almost impossible task.
In contrast, ConfD's data model centric validation concept, i.e. checking the validity of the configuration
that will be the result of those actions, allows for clear and concise validation code that rejects an invalid
configuration regardless of which commands or other operations that were used to (attempt to) create it.
Attempting to validate the operations instead of the resulting configuration can also lead to problems with
loading config backups or doing rollbacks. The old configuration that should be applied as a result of such
actions is obviously valid (as long as the logical constraints have not changed), but validation logic that
rejects specific changes to the configuration may still result in that configuration being rejected.
An additional benefit of the "logical constraints" approach to validation is the possibility of "off-line"
validation - i.e. a complete configuration can be loaded onto a device that is not in production use, and the
validation code can give its verdict about its validity, even though the sequence of operations that would
lead to this configuration may not even be known.
Since ConfD provides access to the complete new configuration inside a transaction for the purpose of
semantic validation, it is generally straightforward to implement constraints that are expressed in terms
of relations between configuration nodes that must hold true for any valid configuration. Identifying and
formulating those constraints is thus the first thing we must do when implementing semantic validation.
There is however one case where using the validation functionality to check operator actions can be useful,
namely when we want to warn the operator of undesirable consequences - e.g. "If you change this value, the
system will reboot". This is not validation in the sense of verifying correctness, since the new configuration
will be valid too and should not be rejected - but the validation code is still the appropriate place to
implement such functionality. To specifically inspect changes from the current configuration to the new
configuration, the MAAPI API provides functions to iterate over all or a subset of the changes, or if CDB
is used, current configuration values can be read via the CDB API.

123

Semantic validation

9.6. Validation Points
In a manner similar to how we use callpoints to register callback functions which read and write data
in external databases, we use named validation points to define which code is responsible for the validation
of different parts of the configuration. However unlike callpoints, validation points do not form a hierarchy
where they "take over" responsibility from validation points higher up in the XML tree.
The validation code can reject the data, accept it, or accept it with a warning. If a warning is produced,
it will be displayed for interactive users (e.g. through the CLI or Web UI). The user may choose to abort
or continue to commit the transaction.
Validation callbacks are typically assigned to individual leafs or containers in the YANG model, but this
is mostly a matter of organization and modularization of the validation code. In some cases it may even
be feasible to use a single validation callback, e.g. on the top level node of the configuration. In such
a case, this callback is responsible for the validation of all values and their relationships throughout the
configuration.
A validation callback is only invoked if its validation point is for an element that exists in the new
configuration. This may be surprising, but it is a logical consequence of the "validate the configuration,
not the operations" concept of ConfD validation - we can not be asked to validate something that does not
exist. Since it sometimes leads to the question "How can I prevent deletion of an element?", an example
may be useful:

container notification {
leaf protocol {
type enumeration {
enum SNMP;
enum SMTP;
enum NETCONF;
}
mandatory true;
}
leaf smtp-server {
type inet:host;
}
}

Here we can configure a notification protocol, and optionally the address of an SMTP server - we want
it to be optional, since it is not needed unless the notification protocol is SMTP. However if protocol
SMTP and a server has been configured, the SMTP server element must not be deleted. But if we assign
a validation point to that element, the callback will not get invoked on deletion, since the element does
not exist in the new configuration!
The solution is in the previous section - we must identify and formulate the logical constraint. In this case
it is "If notification protocol SMTP is configured, an SMTP server must also be configured".
The easiest way to enforce this is through an XPath expression as in:

container notification {
leaf protocol {
must ". != 'SMTP' or ../smtp-server" {
error-message "Must specify smtp-server";
}

124

Semantic validation

type enumeration {
......

Alternatively, if we us C code logic to achieve the same thing to make sure that our callback is invoked,
we assign the validation point to the "protocol" element. The validation callback will then get the value of
this element as a parameter, and the implementation of the constraint check will just be:

if (CONFD_GET_ENUM_VALUE(newval) == nsprefix_SMTP &&
maapi_exists(maapi_socket, tctx->thandle, "/notification/smtp-server") != 1) {
confd_trans_seterr(tctx, "SMTP server must be configured");
return CONFD_ERR;
}

See the examples below for the API details. Note well that since it is based on the logical constraint,
this single expression also covers the other required case of "operational validation", i.e. setting of the
notification protocol to SMTP - it will be rejected unless an SMTP server has been configured.

Note
Validation will always fail if no code is registered under a validation point that would otherwise
have had its callback invoked during validation.

9.7. Validating Data in C
Next we describe how to connect user defined C code to the validation process. We start off with a really
simple YANG model.
module mtest {
namespace "http://tail-f.com/ns/example/mtest";
prefix mtest;
container mtest {
leaf a_number {
type int64;
default 42;
}
leaf b_number {
type int64;
default 7;
}
}
}

We wish to ensure that the integer value /mtest/a_number is bigger than /mtest/b_number. We use a YANG
model annotation file to specify the validation point. This is a good technique to use if wish to keep out
data models clean of any Tail-f extensions.
module mtest.annot {
namespace "http://tail-f.com/ns/example/mtest.annot";
prefix mtesta;
import mtest {
prefix m;
}

125

Semantic validation

import tailf-common {
prefix tailf;
}
tailf:annotate "/m:mtest/m:a_number" {
tailf:validate vp1;
}
}

We define a validation point on /mtest/a_number called vp1. This instructs ConfD that whenever ConfD
needs to validate the XML data element associated with /mtest/a_number, ConfD should call an external
process which has registered itself under the validation point vp1 using the libconfd interface. If no
process is registered, the validation will fail and it will not be possible to commit any changes.
We continue with the necessary C code to implement the relevant parts of the external process. The
complete code for this can be found in examples.confd/validate/c in the ConfD examples
collection. The validation code will be called during the validation phase of a ConfD transaction, i.e. before
any write operations have been performed. The C code must install three different callback functions.
• init() - This callback will be invoked for any transaction where one of our validate() callbacks is
invoked. The purpose of the callback is to initialize data structures and sockets for the remainder of the
transaction. In particular it must indicate to the library which socket should be used for this transaction.
Another task to perform in the init() callback is to attach a MAAPI socket to this transaction. While
validating the individual values, we use MAAPI to possibly read other XML data elements from the
same transaction we are validating. The function maapi_attach() is used to attach our MAAPI
socket to the running transaction.
• validate() - We may have several validation points. We must install one callback for each defined
validation point. The callback will be automatically called by ConfD during the actual validation phase.
The callback must return CONFD_OK if the validation succeeds, CONFD_ERR on validation failure, or
CONFD_VALIDATION_WARN to accept with a warning.
• stop() - This callback will be invoked when the transaction finishes, if the init() callback was
invoked. It will be called regardless of the outcome of the transaction.
The
init()
and
stop()
callbacks
are
installed
through
the
API
call
confd_register_trans_validate_cb() and the individual validation point callbacks are
installed through consecutive calls to confd_register_valpoint_cb() for each defined
validation point.
We start by creating three sockets to ConfD. We need two sockets for the callback machinery and one
MAAPI socket.
#include "confd_lib.h"
#include "confd_dp.h"
#include "confd_maapi.h"
/* include generated ns file */
#include "mtest.h"
int debuglevel = CONFD_DEBUG;
static int ctlsock;
static int workersock;
static int maapi_socket;

126

Semantic validation

static struct confd_daemon_ctx *dctx;

confd_init("more_a_than_b", stderr, debuglevel);
if ((dctx = confd_init_daemon("mydaemon")) == NULL)
confd_fatal("Failed to initialize confd\n");
if ((ctlsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open ctlsocket\n");
addr.sin_addr.s_addr = inet_addr("127.0.0.1");
addr.sin_family = AF_INET;
addr.sin_port = htons(CONFD_PORT);
OK(confd_load_schemas((struct sockaddr*)&addr, sizeof(struct sockaddr_in)));
/* Create the first control socket, all requests to */
/* create new transactions arrive here */
if (confd_connect(dctx, ctlsock, CONTROL_SOCKET,
(struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0) {
confd_fatal("Failed to confd_connect() to confd \n");
}
/* Also establish a workersocket, this is the most simple */
/* case where we have just one ctlsock and one workersock */
if ((workersock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open workersocket\n");
if (confd_connect(dctx, workersock, WORKER_SOCKET,(struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");
if ((*maapi_sock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open socket\n");
if (maapi_connect(*maapi_sock, (struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");

The above code connects three times. We need the control socket and the worker socket for the C callbacks.
This works precisely the same way as when C callbacks are installed for the external data provider API.
Thus the request from ConfD to invoke the init() callback will arrive on the control socket whereas
the subsequent requests to invoke the individual validate() callbacks as well as the finishing request
to invoke stop() will arrive on the designated worker socket.
The MAAPI socket will be used to attach the running transaction to a MAAPI socket.
All three sockets are connected to the same port number.
Next step is to continue with the installation of the callbacks:

struct confd_trans_validate_cbs vcb;
struct confd_valpoint_cb valp1;
static void OK(int rval)

127

Semantic validation

{
if (rval != CONFD_OK) {
fprintf(stderr, "more_a_than_b.c: error not CONFD_OK: %d : %s \n",
confd_errno, confd_lasterr());
abort();
}
}
vcb.init = init_validation;
vcb.stop = stop_validation;
confd_register_trans_validate_cb(dctx, &vcb);
valp1.validate = validate;
strcpy(valp1.valpoint, "vp1");
OK(confd_register_valpoint_cb(dctx, &valp1));
OK(confd_register_done(dctx));

Note the call to confd_register_done() after the callback registrations - this is required, to tell
ConfD that we have completed our registrations. The actual callbacks look like:
static int init_validation(struct confd_trans_ctx *tctx)
{
OK(maapi_attach(maapi_socket, mtest__ns, tctx));
confd_trans_set_fd(tctx, workersock);
return CONFD_OK;
}
static int stop_validation(struct confd_trans_ctx *tctx)
{
OK(maapi_detach(maapi_socket, tctx));
return CONFD_OK;
}
static int validate(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath,
confd_value_t *newval)
{
int64_t b_val;
int64_t a_val;
int64_t newval_a;
/* we validate that a_number > b_number

*/

newval_a = CONFD_GET_INT64(newval);
/* this switch is not necessary in this case; we know that we're
called for a_number only. the switch is useful when the same
code is used to validate multiple objects. */
switch (CONFD_GET_XMLTAG(&(keypath->v[0][0]))) {
case mtest_a_number:
OK(maapi_get_int64_elem(maapi_socket, tctx->thandle, &b_val,
"/mtest/b_number"));
OK(maapi_get_int64_elem(maapi_socket, tctx->thandle, &a_val,
"/mtest/a_number"));
/* just an assertion to show that newval == /mtest/a_number */
/* in this transaction */

128

Semantic validation

assert(CONFD_GET_INT64(newval) == a_val);
if (newval_a == 88) {
/* This is how we get to interact with the CLI/webui */
confd_trans_seterr(tctx, "Dangerous value: 88");
return CONFD_VALIDATION_WARN;
}
else if (newval_a > b_val) {
return CONFD_OK;
}
else {
confd_trans_seterr(tctx, "a_number is <= b_number ");
return CONFD_ERR;
}
break;
default: {
char ebuf[BUFSIZ];
sprintf(ebuf, "Unknown tag %d",
CONFD_GET_XMLTAG(&(keypath->v[0][0])));
confd_trans_seterr(tctx, ebuf);
return CONFD_ERR;
} /* default case */
} /* switch */
}

The switch on the keypath exemplifies that we really get the keypath populated with the path leading to
the textual element being validated. We can thus have the same validation point validate different XML
data elements.
The init callback attaches to MAAPI and the global variable maapi_socket is used to read data
from the transaction. All MAAPI functions use a "transaction handle"; this handle is available inside the
instantiated struct confd_trans_ctx *tctx structure.
Finally we have a poll loop where we dispatch requests to invoke C callbacks on the control socket and
the worker socket.

while (1) {
struct pollfd set[2];
int ret;
set[0].fd = ctlsock;
set[0].events = POLLIN;
set[0].revents = 0;
set[1].fd = workersock;
set[1].events = POLLIN;
set[1].revents = 0;
if (poll(&set[0], 2, -1) < 0) {
perror("Poll failed:");
continue;
}
if (set[0].revents & POLLIN) {
if ((ret = confd_fd_ready(dctx, ctlsock)) == CONFD_EOF) {

129

Semantic validation

confd_fatal("Control socket closed\n");
} else if (ret == CONFD_ERR && confd_errno != CONFD_ERR_EXTERNAL) {
confd_fatal("Error on control socket request\n");
}
}
if (set[1].revents & POLLIN) {
if ((ret = confd_fd_ready(dctx, workersock)) == CONFD_EOF) {
confd_fatal("Worker socket closed\n");
} else if (ret == CONFD_ERR && confd_errno != CONFD_ERR_EXTERNAL) {
confd_fatal("Error on worker socket request\n");
}
}
}

In the above example we also showed how to issue a warning as opposed to a validation failure. Hadn't it
been for that, it would have been considerably easier to express the same validation as an XPath expression.
Thus we attach a must statement to the mtest container as:
container mtest {
must "a_number > b_number" {
error-message "a_number is <= b_number";
}

9.8. Validation Points and CDB
When CDB first starts or upgrades the database it creates a special transaction which, when committed,
will invoke validation. An external validation point (written e.g. in C) has to be registered before these
transactions are committed, otherwise starting ConfD will fail. Starting ConfD and external applications
in a synchronized way is accomplished using ConfD start phases (see the Advanced Topics chapter). To
avoid this extra complexity use the --ignore-initial-validation option when starting ConfD
(useful during development).
In a validation point it might be desirable to access CDB to validate a value against the old value of the
parameter (or some other parameter) in the configuration. Using the normal cdb calls this works fine in
the normal case, but when CDB is initializing there are no old values. The cdb_get_phase() call can
be used to check for this case, (see the confd_lib_cdb(3) manual page for details).

Note
If a CDB session is used throughout the validation phase (i.e. the session is not ended
until the stop() callback invocation), we must start it without a read lock, i.e. using
cdb_start_session2() with flags = 0. It is safe to do that in this particular case, since
the transaction lock prevents changes to CDB during validation.

9.9. Dependencies - Why Does Validation
Points Get Called
In general, validation code for a particular element in the configuration may read any other part of the
configuration, and accept or reject the configuration based on that. I.e. the outcome of the validation may

130

Semantic validation

actually depend on other configuration elements than the one the validation point is assigned to - and for
correct operation, the validation code must be executed when any element it depends on has been modified.
As ConfD can not make any assumptions about these dependencies, it takes the safe default of always
invoking all callbacks (for existing elements) on every configuration change.
It is possible to declare these dependencies explicitly in the YANG model. This can be a significant
optimization, but it is strictly an optimization, i.e. a validation callback implementing a logical constraint
verification will always return the same result for a given configuration, it doesn't matter if it is invoked
unnecessarily due to lack of a dependency declaration in the YANG model. On the other hand an
incorrect dependency declaration, that omits some dependency, can allow changes that lead to an invalid
configuration. Thus if dependency declarations are used, it is critical that they are correct, and in particular
that they are updated as needed if the validation logic of the callback is changed.
There can be multiple dependency declarations for a validation point. Each declaration consists of a
dependency element specifying a configuration subtree that the validation code is dependent upon.
If any element in any of the subtrees is modified, the validation callback is invoked. A subtree can be
specified as an absolute path or as a relative path.
The relative path '.' is often used to declare that the validation code needs to be run whenever the current
element or an element below it is modified. However note that per above, routinely specifying '.' as the only
dependency for all validation points is a dangerous practice - if the validation logic actually depends on
elements outside the subtree of the validation point, an invalid configuration may go undetected. Also, for
a leaf element, having '.' as the only dependency is almost always wrong - if the validation really depends
only on the leaf itself, it is likely that it could be expressed as a constraint in the YANG model instead
of via a validation callback.
As described above, if a dependency is not declared, it defaults to a single dependency on the root of the
configuration tree (/), which means that the validation code is executed when any configuration element
is modified.
If dependencies are declared on a leaf element, an implicit dependency on the leaf itself is added.
As an example, consider the /mtest/a_number validation above. The element a_number has validation
code attached to it, and this code depends on element b_number. Thus, this code has to be executed
whenever a_number or b_number is modified. To specify this, we can do:
leaf a_number {
type int64;
default 42;
tailf:validate vp1 {
tailf:dependency '../b_number';
}
}

Here we specified the validation point with its dependency sub-element directly in the YANG model
- it is of course possible to use an annotation file in this case too.
It is also possible, and recommended for performance reasons, to specify dependencies in must
statements:
leaf a_number {
type int64;
default 42;
must ". > ../b_number" {

131

Semantic validation

tailf:dependency '../b_number';
}
}

The compiler gives a warning if a must statement lacks a tailf:dependency statement, and
it cannot derive the dependency from the expression. The options --fail-on-warnings or -E
TAILF_MUST_NEED_DEPENDENCY can be given to force this warning to be treated as an error.

9.10. Configuration Policies
Configuration policies is an optional mechanism by which the operator of a ConfD-based system can
define its own custom validation rules. A configuration policy enforces custom validation rules on the
configuration data. These rules assert that the user-defined conditions are always true in committed data.
If a configuration change is done such that a policy rule would evaluate to false, the configuration change
is rejected by the system.
As an example, an operator might define a configuration policy that bgp must never be disabled on a
device, or define a policy that the MTU on SONET interfaces must be greater than 2048.
The data model for configuration policies is defined in tailf-configuration-policy.yang, in
the directory $CONFD_DIR/src/confd/configuration_policy/. The YANG model contains
a Also included in this directory is a pre-compiled .fxs file, and a Makefile that can be modified as
necessary, for example to compile the fxs file with a --export parameter to confdc.
To enable this optional feature, put the tailf-configuration-policy.fxs in the load path to
ConfD.

9.10.1. Example
These
examples,
and
more,
are
available
configuration_policy in the distribution.

in

examples.confd/validate/

As a first simple example, we define a policy that makes sure that BGP is always enabled on the box. The
example assumes the following data model:

container protocols {
container bgp {
presence "enables bgp";
// BGP config goes here...
}
}

The following CLI commands, define the policy we need:

admin@host% configure
admin@host% set policy rule chk-bgp expr "/protocols/bgp"
admin@host% set policy rule chk-bgp error-message "bgp must be enabled"
admin@host% commit
Commit complete.

132

Semantic validation

Now, let's try to disable bgp:
admin@host% delete protocols bgp
admin@host% commit
Aborted: bgp must be enabled

As another example, we define a policy that ensures that the MTU of SONET interfaces are greater than
or equal than 2048:
admin@host> set autowizard false
admin@host> configure
admin@host% set policy rule chk-sonet-mtu
admin@host% edit policy rule chk-sonet-mtu
admin@host% set foreach "/interface[type='sonet']"
admin@host% set expr "mtu >= 2048"
admin@host% set error-message "Sonet interface {name} has MTU {mtu}, must be at least 2048"
admin@host% top
admin@host% commit
Commit complete.
[ok][2010-11-02 09:39:00]

This rule uses the foreach leaf. When ConfD evaluates this rule, it will first evaluate the foreach
expression. This expression evaluates to a node set with all sonet interfaces. Then, foreach node in this node
set, the expr expression is evaluated. If it evaluates to false, validation fails with the error message given.
The error message uses a special notation {}. Before the error message is
printed, ConfD substitutes all XPath expressions within { }, by converting the result to a string. In this
example, there are two such XPath expressions {name} and {mtu}.
This is best shown in an example:
mbj@x15% set interface so-1/0 type sonet mtu 4096
mbj@x15% set interface so-1/1 type sonet mtu 4096
mbj@x15% set interface so-1/2 type sonet mtu 1024
mbj@x15% validate
Failed: Sonet interface so-1/2 has MTU 1024, must be at least 2048

133

Chapter 10. Transformations, Hooks,
Hidden Data and Symlinks
10.1. Introduction
When building new variants of an old product, we often have a situation where we have large amounts
of application code, which reads configuration data from some datastore and we do not want to make any
changes to the application code, we merely wish to expose a different view of the same configuration data.
Another common situation is when we have application code which requires more configuration data than
we wish to expose through the northbound management interfaces. The application reads and use a number
of configuration items that do not make sense to expose through the different management interfaces.
In general, when the actual configuration data differs from what we wish to expose as management data,
we can use a variety of different techniques in ConfD.
In this chapter we describe the following four different techniques that all are used to somehow show
different views of the system.
• Transforms are used to show a portion of the system in a different way than the original data model.
• Hooks are used to execute user code whenever a part of the configuration is changed.
• Hidden Data is means to hide parts of the configuration.
• Symlinks are pointers in the data model effectively making one part of the data model appear at a different
place.
Some or several of the above features are often the required trick when we wish to accomplish certain
modifications of the data model. However, combining these features, can make the system complex. A
clean YANG data model is easy to understand - whereas a system consisting of hidden transformations with
symlinks here and there, can very easy become hard to understand. So - use these features with caution.

10.2. Transformation Control Flow
ConfD uses the data model, i.e. the YANG files to render all the northbound interfaces, the layout of the
datamodel is exactly reflected in the CLI and the Web UI. Thus, unfortunately, we may sometimes be
forced to manipulate the data model in order to have a desired look and feel in the CLI or the Web UI. In
these cases, a transform may be required trick.
A transformation works as follows. We start out with the YANG model we wish to transform. We may
add tailf:export maapi to that YANG module. This makes the YANG model invisible to all management
agents except MAAPI. Although invisible, the YANG model is loaded into the system and regardless of
whether the YANG model is populated through cdb or an external database it is fully operational and
accessible through the MAAPI APIs as well as through the CDB APIs (assuming the YANG model is
populated by CDB).
Following that, we write another YANG model which represents the management data we do wish to
expose to the northbound management agents. This YANG model has a callpoint which uses the attribute
tailf:transform. Finally we must write a program which acts as an external database, serving data

134

Transformations, Hooks,
Hidden Data and Symlinks
for a callpoint. This program should use the MAAPI API to read and write the real YANG model data
which is used by the managed objects.

10.3. An Example
Assume we have the following YANG model:

Example 10.1. full.yang
container full {
leaf firstname {
type string;
default George;
}
leaf a_number {
type int64;
default 42;
}
leaf b_number {
type int64;
default 7;
}
container servers {
list server {
key name;
max-elements 64;
leaf name {
type string;
}
leaf ip {
type inet:host;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
}
}
}

For some reason we think that this YANG model is way too complicated to expose through the management
interfaces. We have also invested time and energy in various applications that read and use precisely this
data and we do not want to change any of those applications.
What we want to do is to expose a YANG model which looks like:

Example 10.2. small.yang
container small {
container servers {
tailf:callpoint transcp {
tailf:transform true;
}

135

Transformations, Hooks,
Hidden Data and Symlinks
list server {
key name;
max-elements 64;
leaf name {
type string;
}
}
}
}

I.e. skip all the first toplevel elements, and skip the ip and the port. We write C code which derives both.
Also note the transformation callpoint.
When ConfD needs to read and write data in the small.yang YANG model, it will use the callpoint
transcp. I.e. it will invoke the installed callback functions for that callpoint. The main difference
between a normal callpoint and a transformation callpoint is when write and when validation occurs. In
a transformation callpoint the write operations occur before validation. This means that any data that is
written through MAAPI in the actual transform, will also be validated.
Similar to callpoints for external data, we need a worker socket a control socket and registered callbacks.
Our main() function together with some global variables would look like:

#include "full.h" /* generated .h files */
#include "small.h"
static
static
static
struct
struct
static

struct confd_daemon_ctx *dctx;
int ctlsock;
int workersock;
confd_trans_cbs tcb;
confd_data_cbs data;
int maapi_socket;

int main()
{
struct in_addr in;
struct sockaddr_in addr;
confd_init("MYNAME", stderr, debuglevel);
if ((dctx = confd_init_daemon("mydaemon")) == NULL)
confd_fatal("Failed to initialize confd\n");
if ((ctlsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open ctlsocket\n");
inet_aton("127.0.0.1", &in);
addr.sin_addr.s_addr = in.s_addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(CONFD_PORT);
confd_load_schemas((struct sockaddr*)&addr,sizeof (struct sockaddr_in))l
if (confd_connect(dctx, ctlsock, CONTROL_SOCKET, (struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");
if ((workersock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open workersocket\n");

136

Transformations, Hooks,
Hidden Data and Symlinks
if (confd_connect(dctx, workersock, WORKER_SOCKET,(struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");

tcb.init = init_transformation;
tcb.finish = stop_transformation;
confd_register_trans_cb(dctx, &tcb);

data.get_elem = get_elem;
data.get_next = get_next;
data.set_elem = set_elem;
data.create
= create;
data.remove
= dbremove;
data.exists_optional = NULL;
strcpy(data.callpoint, "transcp");
if (confd_register_data_cb(dctx, &data) != CONFD_OK)
confd_fatal("Failed to register data cb \n");
if (confd_register_done(dctx) != CONFD_OK)
confd_fatal("Failed to complete registration \n");
setup_maapi_sock(&maapi_socket));
.........

The above is precisely the same setup as when we register callbacks for an external database with the only
exception that the callpoint uses tailf:transform true.
The code to establish the MAAPI socket is just a call to maapi_connect().
The difference comes in the implementation of the data callbacks, with an external database, we have the
data to deliver, in the case of a transformation callpoint, we don't have the data. The data resides inside
ConfD and we can read and write that data inside the same transaction using maapi_attach().
The initialization looks like:

static int init_transformation(struct confd_trans_ctx *tctx)
{
maapi_attach(maapi_socket, full__ns, tctx);
confd_trans_set_fd(tctx, workersock);
return CONFD_OK;
}
static int stop_transformation(struct confd_trans_ctx *tctx)
{
if (tctx->t_opaque != NULL) {
struct maapi_cursor *mc = (struct maapi_cursor *)tctx->t_opaque;
maapi_destroy_cursor(mc);
free(tctx->t_opaque);
}
maapi_detach(maapi_socket, tctx);
return CONFD_OK;
}

137

Transformations, Hooks,
Hidden Data and Symlinks
Whenever a transaction starts, we get called - as usual - in our init() callback and whenever the
transaction terminates, regardless of the outcome of the transaction, we get called in our finish()
callback. Here we attach to the executing transaction using maapi_attach() in the init() callback.
We will use the attached MAAPI socket with the right transaction handle in all our data processing
callbacks. In the finish() callback we need to release any memory used for a MAAPI cursor, see
get_next() below.
The get_elem() callback is interesting. The path we get queried with is /small/servers/
server{key}/name which doesn't exist in the real database. What does exist as proper data though
is the path /full/servers/server{key}/name and we use the MAAPI socket to read that value
from the "hidden" YANG model.

static int get_elem(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
confd_value_t v;
confd_value_t *leaf = &(keypath->v[0][0]);
confd_value_t *vp = &(keypath->v[1][0]);
switch (CONFD_GET_XMLTAG(leaf)) {
case small_name:
if (maapi_get_elem(maapi_socket, tctx->thandle, &v,
"/full/servers/server{%x}/name",
confd_data_reply_value(tctx, &v);
free(CONFD_GET_BUFPTR(&v));
return CONFD_OK;
}
else if (confd_errno == CONFD_ERR_NOEXISTS) {
fprintf(stderr, "\nNOT FOUND \n");
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
else {
fprintf (stderr, "errno = %d\n", confd_errno);
return CONFD_ERR;
}
default:
return CONFD_ERR;
}

vp) == CONFD_OK) {

}

It is important that we check confd_errno to distinguish between the real error cases and the case
where the element doesn't exist. Also note how we format the path to the maapi_get_elem() call
using the second element in the keypath. This will be the key since the path will be /small/servers/
server{key}/name.
set_elem() will never be called since our MO server only contains a key and no other elements.
The callback get_next() is also interesting. Here we utilize a MAAPI cursor to iterate through the
different "full" servers. Since the MAAPI cursor must exist across calls to get_next(), and we must
be able to handle multiple transactions (from different user sessions) in parallel, we allocate the cursor
dynamically and use the t_opaque element in the transaction context to keep track of it.

static int get_next(struct confd_trans_ctx *tctx,

138

Transformations, Hooks,
Hidden Data and Symlinks
confd_hkeypath_t *keypath,
long next)
{
struct maapi_cursor *mc;
if (next == -1) {
if (tctx->t_opaque == NULL) {
/* allocate the cursor */
mc = (struct maapi_cursor *)malloc(sizeof(struct maapi_cursor));
tctx->t_opaque = mc;
} else {
/* re-use previously allocated cursor */
mc = (struct maapi_cursor *)tctx->t_opaque;
maapi_destroy_cursor(mc);
}
maapi_init_cursor(maapi_socket, tctx->thandle, mc,
"/full/servers/server");
} else {
mc = (struct maapi_cursor *)tctx->t_opaque;
}
maapi_get_next(mc);
if (mc->n == 0) {
confd_data_reply_next_key(tctx, NULL, -1, -1);
return CONFD_OK;
}
confd_data_reply_next_key(tctx, &(mc->keys[0]), 1, 1);
return CONFD_OK;
}

Finally we have the delete() and create() callbacks. The delete() callback is completely
straightforward where we simply delete the same element from the hidden "full" YANG model. The
create() callback needs to do a bit of work. The "full" YANG model contains two elements that are
not part of the "small" YANG model. Thus when a manager creates a new element in the "small" YANG
model, it is the responsibility of our code here to create the corresponding element in the "full" YANG
model, but also to populate the additional two elements with values. If we fail to do that the commit will
fail since values in the "full" YANG model are unset.
The code to delete and create:

static int dbremove(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
maapi_delete(maapi_socket, tctx->thandle,
"/full/servers/server{%x}", &(keypath->v[0][0]));
return CONFD_OK;
}

static int create(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
/* this is where we have to do extra, we need to also populate */
/* the ip and ports fields in full.cs */
char buf[BUFSIZ];
confd_value_t *key = &(keypath->v[0][0]);
struct servent *srv;
maapi_create(maapi_socket, tctx->thandle,

139

Transformations, Hooks,
Hidden Data and Symlinks
"/full/servers/server{%x}", key);
maapi_set_elem2(maapi_socket, tctx->thandle,
"0.0.0.0",
"/full/servers/server{%x}/ip",key);
/* NUL terminate string */
memcpy(buf, CONFD_GET_BUFPTR(key), CONFD_GET_BUFSIZE(key));
buf[CONFD_GET_BUFSIZE(key)] = 0;
if ((srv = getservbyname(buf, NULL)) == NULL) {
char tbuf[BUFSIZ];
sprintf(tbuf, "Unknown service %s", buf);
confd_trans_seterr(tctx, tbuf);
return CONFD_ERR;
}
sprintf(buf, "%d", srv->s_port);
maapi_set_elem2(maapi_socket, tctx->thandle, buf,
"/full/servers/server{%x}/port",key);
return CONFD_OK;
}

10.4. AAA Transform
The ConfD AAA YANG model is a very good example of where we may wish to expose a different set
of configuration items to the management stations than what exists in the AAA YANG model tailfaaa.yang. The AAA system is described in Chapter 14, The AAA infrastructure.
The data in the AAA YANG model is used by ConfD itself and all that data including the fairly complicated
authorization rules must be there for ConfD to read. We think that very few devices wish to expose e.g. the
authorization rules from tailf-aaa.yang to end users. The solution to this is to use a transformation.
In the ConfD examples collection, we have an example which exposes a very simple AAA model. The
simple AAA YANG model looks as:

Example 10.3. users.yang
module users {
namespace "http://www.example.com/ns/users";
prefix u;
import tailf-common {
prefix tailf;
}
typedef Role {
type enumeration {
enum admin;
enum oper;
}
}
typedef passwdStr {
type tailf:md5-digest-string {
}
}
container users {
tailf:callpoint simple_aaa {
tailf:transform true;

140

Transformations, Hooks,
Hidden Data and Symlinks
}
list user {
key name;
max-elements 64;
leaf name {
type string;
}
leaf password {
type passwdStr;
mandatory true;
}
leaf role {
type Role;
mandatory true;
}
}
}
}

This YANG model just exposes a list of users. Each user has a password and an enum indicating the role
of the user. There are only two static roles to choose from, admin and oper.
If we also have intimate knowledge of the datamodel we are using, it is possible to generate static
authorization rules.
Let's take a look at the get_elem() callback. The task of this callback is to use MAAPI in order to
populate the simple YANG model from above.

static int get_elem(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath)
{
confd_value_t v;
confd_value_t *leaf = &(keypath->v[0][0]);
confd_value_t *vp = &(keypath->v[1][0]);
switch (CONFD_GET_XMLTAG(leaf)) {
case aaa_simple_name:
if (maapi_get_elem(
maapi_socket, tctx->thandle, &v,
"/aaa/authentication/users/user{%x}/name", vp) == CONFD_OK) {
confd_data_reply_value(tctx, &v);
free(CONFD_GET_BUFPTR(&v));
return CONFD_OK;
}
else if (confd_errno == CONFD_ERR_NOEXISTS) {
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
else {
printf ("errno = %d\n", confd_errno);
return CONFD_ERR;
}
case aaa_simple_password:
if (maapi_get_elem(
maapi_socket, tctx->thandle, &v,
"/aaa/authentication/users/user{%x}/password", vp)==CONFD_OK) {

141

Transformations, Hooks,
Hidden Data and Symlinks
confd_data_reply_value(tctx, &v);
free(CONFD_GET_BUFPTR(&v));
return CONFD_OK;
}
else if (confd_errno == CONFD_ERR_NOEXISTS) {
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
else {
fprintf (stderr, "errno = %d\n", confd_errno);
return CONFD_ERR;
}
case aaa_simple_role: {
int ret;
char users[BUFSIZ];
char user[256];
memcpy(&user[0], CONFD_GET_BUFPTR(vp), CONFD_GET_BUFSIZE(vp));
user[CONFD_GET_BUFSIZE(vp)] = 0;
ret = maapi_get_str_elem(
maapi_socket, tctx->thandle, users, BUFSIZ,
"/aaa/authentication/groups/group{admin}/users");
if (strstr(users, user) != NULL) {
CONFD_SET_ENUM_VALUE(&v, aaa_simple_admin);
confd_data_reply_value(tctx, &v);
return CONFD_OK;
}
else {
maapi_get_str_elem(
maapi_socket, tctx->thandle, users, BUFSIZ,
"/aaa/authentication/groups/group{oper}/users");
if (strstr(users, user) != NULL) {
CONFD_SET_ENUM_VALUE(&v, aaa_simple_oper);
confd_data_reply_value(tctx, &v);
return CONFD_OK;
}
}
/* user not part of any group at all */
confd_data_reply_not_found(tctx);
return CONFD_OK;
}
default:
confd_fatal("Unexpected switch tag %d\n",
CONFD_GET_XMLTAG(leaf));
}
return CONFD_ERR;
}

10.5. Other Use Cases for Transformations
We may also envision a use case where we wish to expose more data than is available in the YANG model.
In this case the task of the transformation would be to aggregate the data and write into MAAPI.
Yet another use case for transformations would be when we wish to expose two variants of the same config,
one for novices and one for experts. In this case we have the full YANG model with all the details exposed
to experts and a simplified version which fills in many reasonable default values and possibly also derives
data, exposed to novices. The YANG model for novices would then be populated by a transformation.

142

Transformations, Hooks,
Hidden Data and Symlinks
One common transformation is to logically move an entire subtree of some data model to some other
place. For example, ConfD's AAA data model is named /aaa, but suppose we want to access it through
/system/advanced/aaa instead. This can be done by using a transform as described above, or it can
be done using a symlink, which essentially is a specialized, built-in transform.
Finally when we want to implement support for standard SNMP mibs while at the same time use a proper
hierarchical high level data model for all other north bound interfaces we must use a transform on the
SNMP data. To implement this we compile the mib into a YANG model document using the smidump
tool. We must then also annotate the YANG model derived from the MIB and set a transformation point
at the top. Thus when the SNMP agent tries to read data from the MIB (through the YANG model) our
transformation C code gets invoked and we can then, over the maapi interface, read the right data from
the high level data model and return that data to the SNMP agent.

10.6. Hooks
A hook is a function that is invoked within the transaction when an object is modified. The hook function
has access to the transaction, so it can modify other objects in the transaction as necessary.
A hook is a way for the application to participate in a transaction. A hook is like a callpoint or a validation
point only that the application gets to attach (using maapi) to the transaction and can write more data.
For example if we have an optional container containing a set of items, whenever the container is created,
our hook gets called and the container can be populated with proper values. This effect is also achieved by
letting the container elements have default values. The "default value" solution is compile time, whereas
a populating the container through a hook is obviously runtime.
Hooks can also be used to attach some magic to individual elements. Say that we have a leaf:

leaf magic {
type int32;
}

Whenever the magic leaf get set to, say -1, our hook code performs some other arbitrary write operations.
Thus the hook mechanism can be used to achieve a wide variety of effects.
A hook is implemented similar to a callpoint with the exception that only write callbacks need to
be implemented. The write callbacks are set_elem(), create(), remove(), set_case(),
set_attr(), and move_after(). However a hook only needs to implement the write callbacks that
it actually needs for its own use - the others can be left unimplemented, indicated by setting them to NULL
in the callback registration.
There are two types of hooks, set hooks and transaction hooks. The main difference between the two is
when they are invoked. Set hooks are invoked directly as an object is modified, and transaction hooks
are invoked in the two-phase commit phase. The ConfD transaction engine receives all the original write
operations from one of the north bound agents. Once all write operations have been received, i.e. when a
user for example types "commit" in the CLI, the transaction engine invokes the relevant transaction hooks.
Once all transaction hooks are run the validation phase is entered, thus the write operations performed by
the transaction hooks are also validated.
When set hooks make changes to the configuration, these changes are just like changes done directly by
the user - they will be committed together with other changes, e.g. when the user types "commit" in the

143

Transformations, Hooks,
Hidden Data and Symlinks
CLI. If such a commit operation cannot be completed, due to validation errors, the changes done by set
hooks will remain in the change set until explicitly reverted.
Changes done by transaction hooks are different, in that ConfD keeps track of them, and rolls them back
in case a commit operation cannot be completed. This makes it possible for the user to fix validation errors
and attempt to commit again.
The hook is specified similar to callpoint if we have:
list dyn {
key name;
max-elements 64;
tailf:callpoint foocp {
tailf:transaction-hook subtree;
}
leaf name {
type string;
}
leaf aval {
type string;
mandatory true;
}
leaf container {
type empty;
}
}

The statement: tailf:transaction-hook subtree; indicates that we wish to attach a hook
to this part of the data model. A set hook uses the statement tailf:set-hook instead. Similar
to a validator, the hook code will participate in the transaction by calling maapi_attach()
and similar to a data provider, the hook code must register its callbacks through calls to
confd_register_trans_cb() and confd_register_data_cb(). Only those of the possible
write callbacks that are actually needed by the hook need to be registered as data callbacks. All other
callbacks must be set to NULL.
For example our set_elem() callback could look like:
static int my_set_elem(struct confd_trans_ctx *tctx,
confd_hkeypath_t *keypath,
confd_value_t *newval)
{
confd_value_t *tag = &(keypath->v[0][0]);
confd_value_t *key = &(keypath->v[1][0]);
if (confd_svcmp("donk", key) == 0) {
maapi_setelem2(maapisock, tctx->thandle,"222", "/foo/bar");
.....

So whenever some north bound agent assigns the value "donk" to /dyn{key}/aval for all values of
key our code kicks in and additionally assigns a value to /foo/bar.
We can have three different kinds of hooks.
1. subtree - this assigns the hook code to all objects found below where the hook is defined. The value
"true" is the same as "subtree".

144

Transformations, Hooks,
Hidden Data and Symlinks
2. object - This is used when we wish to assign a hook to the manipulation of list entries. The hook
reaches down to and including the list where it is defined. If there exists further lists further down in
the tree they are not affected by the hook.
3. node - This is used when we wish to assign a hook an optional container and only that. It affects the
container but non of its children.
In some cases a transaction hook may need to update the transaction in a way that really depends
on the complete configuration, rather than on the changes done in the current transaction, making it
difficult to implement the hook via the create(), set_elem(), etc callbacks. We can then use the
tailf:invocation-mode substatement to tailf:transaction-hook, like this:

tailf:callpoint foocp {
tailf:transaction-hook subtree {
tailf:invocation-mode per-transaction;
}
}

The per-transaction argument tells ConfD that this hook should only have one data callback
invocation, regardless of the details of the changes to the objects the hook is assigned to. We can even use
the same callpoint name, with the same tailf:invocation-mode statement, at several points in the
data model, and still only get one callback invocation. The data callback that gets invoked for a transaction
hook specified like this is called write_all() (see the confd_lib_dp(3) manual page). It is thus the
only callback that should be registered for such a hook.
It is possible to define multiple hooks for a given node in the data model, and have them all invoked.
However hooks on descendant nodes override all hooks that are inherited from ancestor nodes. To
"accumulate" hooks, the inherited hooks that should remain in effect need to be specified again on
descendant nodes that have hooks. Consider the following example:

list dyn {
key name;
tailf:callpoint t1 {
tailf:transaction-hook subtree;
}
tailf:callpoint s1 {
tailf:set-hook subtree;
}
leaf name {
type string;
}
leaf aval {
type string;
}
leaf bval {
type string;
tailf:callpoint t2 {
tailf:transaction-hook subtree;
}
}
leaf cval {
type string;
tailf:callpoint t1 {
tailf:transaction-hook subtree;
}

145

Transformations, Hooks,
Hidden Data and Symlinks
tailf:callpoint s2 {
tailf:set-hook subtree;
}
}
}

The result of this will be that changes to the dyn list node or the aval leaf will invoke the callbacks
registered for the t1 and s1 callpoints, while changes to the bval leaf will only invoke the callbacks
registered for the t2 callpoint, and changes to the cval leaf will invoke the callbacks registered for the
t1 and s2 callpoints.
Since hook code gets to execute for all the possible write callbacks, the number of use cases for hook code
is very large. One common use case is once again associated to the implementation of standard MIBS.
Depending on the nature of the chosen standard MIBs, we may need to maintain mapping tables. If for
example the keys differ in the SNMP table from the high level data model we may need to maintain
additional mapping tables that are maintained by hook code.

10.6.1. Set Hooks and Candidate Configuration
When ConfD has been configured to provide a candidate configuration, set hook code will be invoked
when changes are done to the candidate configuration, while transaction hooks will be invoked when the
candidate is committed to running.
There are situations when you only want your hook to modify the running configuration:
• A hook can use maapi to modify config elements that the operator is not allowed to modify directly,
according to the active aaa rule set. If such a modification is done on the candidate configuration store,
the operator will not be allowed to commit the candidate configuration to the running configuration. In
this situation you must thus use a transaction hook to modify the configuration.

10.7. Hidden Data
It is sometimes useful to hide nodes from some of the northbound interfaces. The tailf:export
statement can be used to hide an entire namespace. More fine grained control can be attained with the
tailf:hidden statement.
The tailf:hidden statement names a hide group, i.e. all containers and leafs that has the tailf:hidden
statement, with a specific hide group, are treated the same way as far as being hidden or invisible. The
hide group name full is given a special meaning. The full hide group is hidden from all northbound
interfaces, not just user interfaces.
A node with the tailf:hidden statement must be optional or have a default value if it can be implicitly
created via the creation of a differently hidden node higher up in the hierarchy (e.g. hidden leafs in a nonhidden list entry).
A related situation is when some nodes should be displayed to user only when a certain condition is met. For
example, the ethernet subtree should be displayed only when the type of an interface is ethernet.
This can be achieved through the tailf:display-when statement.

10.7.1. Fully Hidden Nodes
This is nodes that may be useful for the application code, but should be hidden from all northbound
interfaces. An example is the set of physical network interfaces on a device and their types. This is static

146

Transformations, Hooks,
Hidden Data and Symlinks
data, i.e. it can't be changed by configuration, but it can vary between different models of a device that run
the same software, and the device-specific data can be provided via init file or through MAAPI.
This type of data could also be realized via a separate namespace where tailf:export is used to limit
the visibility, but being able to have some nodes in the data model hidden while others are not allows
for greater flexibility - e.g. list entries in the config data can have hidden containers or leafs, which get
instantiated automatically along with the visible config nodes.

10.7.2. Hiding nodes from User Interfaces
This is data that is fully visible to programmatic northbound interfaces such as NETCONF, but normally
hidden from user interfaces such as CLI and Web UI. Examples are data used for experimental or endcustomer-specific features, similar to hidden commands in the CLI but for data nodes.
A user interface may give access to this type of data (and even totally hidden data) if the user executes an
unhide command identifying the set of hidden data that should be revealed. After this these data nodes
appear the same as unhidden data, i.e. they are included in tab completion, listed by show commands etc.
A hide group can only be unhidden if the group is listed in the confd.conf file. This means that a hide
group will be completely hidden to the user interfaces unless it has been explicitly allowed to be unhidden
in the confd.conf file. A password can optionally be required to unhide a group.

debug
secret


10.7.3. Conditional Display
A typical usage example is a discriminated union. One leaf is the type of something, and depending on
the value of this leaf, different containers are visible:
container service {
leaf type {
default http;
type enumeration {
enum http;
enum smtp;
}
}
choice service-type {
container http {
presence "HTTP enabled";
tailf:display-when '/service/type = "http"';
leaf addr {
mandatory true;
type inet:ipv4-address;
}
leaf docroot {
mandatory true;
type string;
}
}
container smtp {

147

Transformations, Hooks,
Hidden Data and Symlinks
presence "SMTP enabled";
tailf:display-when '/service/type = "smtp"';
leaf smtp-relay {
mandatory true;
type boolean;
}
leaf use-virtual-mbox {
type boolean;
}
}
}
}

In this example, the "smtp" container should be visible to the user only when the value of servicetype is smtp.
This can be accomplished by using the tailf:display-when statement. It contains an XPath
expression which specifies when the node should be displayed:

10.8. tailf:symlink
NOTE: The usage of symlink is not recommended. If it is used, use it carefully. It can be made to work in
some special cases, but some other use cases, described below, are problematic. In the problematic cases,
tailf:link can often be used instead. See Section 10.8.1, “Discussion” for more details.
Sometimes we want to move things in our data models. One major downside of moving structures around
in the data model is that all code that reads data from CDB has to be changed. Sometimes this is not feasible
because we do not want to change this code. In this case the symlink feature may be a solution.
Another situation where symlinks might be the remedy is when we want to move things in the data model
due to aesthetics in the human interfaces, the CLI and the Web UI that are both rendered from the data
model.
The syntax for a symlink is straightforward:
container top {
tailf:symlink foo {
tailf:path "/baz/bar";
}
}
container baz {
container bar {
leaf target {
type string;
default "Zappa";
}
}
}

With the above construct we create a link from /top/foo to /baz/bar.
The net result of the link is that it will appear as if all data found in the data model below the link target,
e.g. /baz/bar will appear under the link source, e.g. /top/foo. Thus if prior to the symlink the path
/baz/bar{13}/server{www}/port was a valid key path, now the path /top/foo/bar{13}/

148

Transformations, Hooks,
Hidden Data and Symlinks
server{www}/port point to the same configuration item. Hence we now have two different ways to
configure the same item. The remedy to that is is typically to hide the the target configuration from the
north bound agents using the tailf:hidden or tailf:export statements.
The textual format of a symlink is an XPath absolute location path.
The target of a symlink can contain instantiated keys. Using XPath notation we could point to the
counters element in a specific server as in :
tailf:symlink mycounter {
tailf:path "/servers/server[ip='10.0.0.1'][port=80]'
+ "/counter";
}

Slightly more advanced is that the target can refer to keys in the source. If the target lies within a list,
all keys must be specified. A key either has a value, or is a reference to a key in the path of the source
element, using the function current() as starting point for an XPath location path. Example of a valid
target for a symlink is:
/servers/server[ip='10.0.0.1'][port=current()/../myport]/mycounter

This feature must be used when we want to use keys in our own path down the XML tree and use those
very same keys to find an other way down the XML tree.
At link time, confdc checks that the target of the symlink can actually exist at run-time. A symlink may
point to another symlink. confdc checks at link time, and ConfD at load time, that no cycles exist. Dangling
pointers may occur though. If we have a symlink that points into an optional container or into a list entry,
the target container can be removed. It is up to the application developers to check that dangling pointers
do not occur. If they do, and the symlink is accessed, this is logged in the developer log as an error.
Finally we need to mention that when we are symlinking into another namespace, we must indicate that
using the default XML notation for referencing elements in other namespaces. Thus if we want to have a
symlink into the http://example.com/ns/servers namespace we typically have to import the
module defining that namespace, and use the prefix of that module in the symlink as in:
import servers {
prefix srv;
}
...
tailf:symlink myserver {
tailf:path "/srv:servers/srv:server";
}

10.8.1. Discussion
Northbound Client Applications
The YANG modules that the device publishes to the northbound clients define the API between the client
and the device. Specifically, the YANG module defines the complete data tree for configuration and
operational state.
One problem with tailf:symlink is that a client application (EMS or NMS or similar) cannot just
ignore the statement, since it defines a subtree in the datastore. But since this is not a standard statement,

149

Transformations, Hooks,
Hidden Data and Symlinks
third-party clients will ignore it. This means that when they talk to the device, they will not understand
the subtree under the symlink.
Note that symlink is used to provide a different entry point to the same underlying data. If both the data
model with the symlink and the target data model is being exposed at the same time to the client, this will
be confusing to the client. A change in one part of the tree makes another part change automatically.
One solution to this problem is to make sure that the module with the symlink is not exposed to the client.
Instead, publish the target module only.
Another solution to this problem is to not publish the target module, but instead use pyang to sanitize the
module with the symlink. The sanitizing process replaces the symlink statement with a copy of the target
subtree. Then publish the sanitized version of the module.
NOTE: If a module is not published to a client, it should have a tailf:export statement, to
exclude the protocol the client is using. For example, suppose we have a symlink in our module acmesystem.yang from /system/netconf/nacm to the standard NACM path /nacm. The standard
NACM module might be annotated with:
tailf:export netconf;

and the system module with:
tailf:export cli;
tailf:export webui;

Errors
Another problem with symlinks is if the target subtree contains must expression or validation code that
refers to nodes outside the target subtree. For example, if a symlink foo points to /x/z in this example:
container x {
leaf y {
type int32;
}
leaf z {
type int32;
must ". > ../y" {
error-message "x must be greater than y";
}
}

If someone sets the foo to a value that happens to invalidate the must expression, the error message will
be misleading since it refers to a node they cannot see, and it is not easily fixed, since they cannot modify
the node y.

150

Chapter 11. Actions
11.1. Introduction
When we want to define operations that do not affect the configuration data store, we can use the
tailf:action statement in the YANG data model. The action definition specifies how the action is
invoked, including input and output parameters (if any). Once defined, the action is available for invocation
from all of NETCONF, CLI and Web UI. The action can be implemented either as a callback function
or as an executable. Action support is also discussed in Chapter 15, The NETCONF Server, Chapter 16,
The CLI agent, and WebUI.

11.2. Action as a Callback
To specify that the action is implemented as a callback in an application Daemon, that registers with
ConfD via the C API described in the confd_lib_dp(3) manual page, we use the tailf:actionpoint
statement.
The application must register the callback by calling this function:
int confd_register_action_cbs(struct confd_daemon_ctx *dx, const struct
confd_action_cbs *acb);
The struct confd_action_cbs is defined as:
struct confd_action_cbs {
char actionpoint[MAX_CALLPOINT_LEN];
int (*init)(struct confd_user_info *uinfo);
int (*abort)(struct confd_user_info *uinfo);
int (*action)(struct confd_user_info *uinfo,
struct xml_tag *name,
confd_hkeypath_t *kp,
confd_tag_value_t *params,
int nparams);
int (*command)(struct confd_user_info *uinfo,
char *path, int argc, char **argv);
int (*completion)(struct confd_user_info *uinfo,
int cli_style, char *token, int completion_char,
confd_hkeypath_t *kp,
char *cmdpath, char *cmdparam_id,
struct confd_qname *simpleType, char *extra);
void *cb_opaque;
/* private user data */
};

The actionpoint element gives the name of the actionpoint from the data model, and the init and
action elements must point to two callback functions that are called in sequence when the action is
invoked. In the init() callback, we must associate a worker socket with the action. This socket will be
used for the invocation of the action() callback, which actually carries out the action. Thus in a multi
threaded application, actions can be dispatched to different threads.
The action() callback is invoked with parameters pertaining to the action, in particular a hashed
Keypath that can identify a particular list instance that the action should be applied to, and an array
giving the input parameters. The parameters have the form of an XML instance document conforming to
the specification in the input statement in the data model, and are represented as described for the Tagged
Value Array format in the section called “XML STRUCTURES” in the confd_types(3) manual page. If the
action should return any data values, it must call confd_action_reply_values() with an array of
values in the same form, conforming to the specification in the output statement in the data model.

151

Actions

Unlike the callbacks for data and validation, there is no transaction associated with an action callback.
However an action is always associated with a user session (NETCONF, CLI, etc), and only one action
at a time can be invoked from a given user session.
See the section called “CONFD ACTIONS” in the confd_lib_dp(3) manual page for additional information
about the action callbacks.

11.2.1. Example
As an example, we will look at one of several actions implemented in intro/7-c_actions in the
ConfD examples collection. The data model defines a list of servers, and an action that allows us to request
that a server is reset at some point in time. First, we specify the action in the data model like this:
list server {
key name;
max-elements 64;
leaf name {
tailf:cli-allow-range;
type string;
}
tailf:action reset {
tailf:actionpoint reboot-point;
input {
leaf when {
type string;
mandatory true;
}
}
output {
leaf time {
type string;
mandatory true;
}
}
}
}

Our implementation must have an init() callback and an action() callback. The init() callback
is straightforward:
static int init_action(struct confd_user_info *uinfo)
{
int ret = CONFD_OK;
printf("init_action called\n");
confd_action_set_fd(uinfo, workersock);
return ret;
}

I.e. it just tells ConfD that we want the previously connected worker socket to be used for the invocation
of the action() callback. The calls to printf(3) in these callback functions are of course only for
the purpose of illustration when we run the example. The action() callback, called do_action(),
has a twist: we are using the same callback for multiple actions in the data model, and the code looks at
the name parameter (the argument to tailf:action in the data model) to decide what to do:
/* This is the action callback function. In this example, we have a
single function for all four actions. */

152

Actions

static int do_action(struct confd_user_info *uinfo,
struct xml_tag *name,
confd_hkeypath_t *kp,
confd_tag_value_t *params,
int n)
{
confd_tag_value_t reply[3];
int i, k;
char buf[BUFSIZ];
char *p;
printf("do_action called\n");
for (i = 0; i < n; i++) {
confd_pp_value(buf, sizeof(buf), CONFD_GET_TAG_VALUE(¶ms[i]));
printf("param %2d: %9u:%-9u, %s\n", i, CONFD_GET_TAG_NS(¶ms[i]),
CONFD_GET_TAG_TAG(¶ms[i]), buf);
}
switch (name->tag) {

Our "reset" action is handled in this branch:
case config_reset:
printf("reset\n");
p = CONFD_GET_CBUFPTR(CONFD_GET_TAG_VALUE(¶ms[0]));
i = CONFD_GET_BUFSIZE(CONFD_GET_TAG_VALUE(¶ms[0]));
strncpy(buf, p, i);
buf[i] = 0;
strcat(buf, "-result");
i = 0;
CONFD_SET_TAG_STR(&reply[i], config_time, buf); i++;
confd_action_reply_values(uinfo, reply, i);
break;

The when leaf from the input statement in the data model is the first and only parameter, and is available
to the callback in params[0]. A real implementation of "reset" would analyze the string, e.g. "now" for
immediate reset or a date and time for some point in the future when the reset should be carried out. Here
we just append "-result" to the string and return that as a reply for the time leaf in the output statement,
by setting the first element in our reply[] array and calling confd_action_reply_values().
Finally we must register the callbacks:
/* register the action handler callback */
memset(&acb, 0, sizeof(acb));
strcpy(acb.actionpoint, "reboot-point");
acb.init = init_action;
acb.action = do_action;
acb.abort = abort_action;
if (confd_register_action_cbs(dctx, &acb) != CONFD_OK)
fail("Couldn't register action callbacks");

11.2.2. Using Threads
If we have long-running action callbacks (e.g. file download), it will typically be necessary to use multithreading for the daemon that handles the callbacks. Without threads, only one invocation of a given action
callback can be running at any point in time. Thus if the same action is requested from another user session,
the request will block until the currently running invocation has completed.

153

Actions

Even if we do not want to allow multiple invocations of an action callback to run in parallel, having one
thread for the control socket and one for the worker socket will make it possible to return an error from
the action init() callback when we are "busy" with a running action, instead of having the user wait
for the currently running action to complete. It will also allow us to handle other control socket requests
promptly. In the general case, where we do want to handle multiple action callback requests in parallel,
we need to use multiple worker sockets, with one thread handling each worker socket. We also need one
thread to handle the control socket, dispatching the callback requests to the different worker sockets.
The strategy to use for creating and allocating the worker sockets and threads is up to the application, based
on the needs for responsiveness in the user interface, resource usage requirements, and other applicationspecific considerations. We can set up a fixed pool of sockets and threads on startup, or we can connect
worker sockets and spawn threads dynamically on demand, as well as close worker sockets and terminate
threads that are no longer in use.
The intro/9-c_threads example in the ConfD examples collection demonstrates one such strategy,
where worker sockets/threads for action callbacks are created on demand, giving each user session that
requests an action its own dedicated action worker thread. The user session stop() callback (see
confd_register_usess_cb() in the confd_lib_dp(3) manual page) is used to mark sockets/threads
as "idle" and available for assignment as action workers for other user sessions. With this strategy we may
need as many threads as there can be concurrent user sessions - by default there is no limit on this, but
such limits can be configured in confd.conf (e.g. /confdConfig/sessionLimits/maxSessions
for the total number of concurrent user sessions across all northbound interfaces), see the confd.conf(5)
manual page.

11.3. Action as an Executable
To specify that the action is implemented as a standalone executable (this could be either a compiled
program or a script), that is run to completion on each action invocation, we use the tailf:exec
statement. This has several substatements specifying how the executable should be invoked - for the full
details, see the tailf_yang_extensions(5) manual page:
tailf:args
A space separated list of argument strings to pass to the executable when it is invoked by ConfD. It
may contain variables on the form $(variablename), that are expanded before the command is
executed. E.g.
tailf:args "-p $(path)";

will result in the first argument being "-p" and the second being the CLI form (space-separated
elements) of the path leading to the action's parent container.
tailf:uid
The user id to use when running the executable.
tailf:gid
The group id to use when running the executable.
tailf:wd
The working directory to use when running the executable. If not specified, the home directory of
the user invoking the action is used, except for the case of a CLI session invoked via the confd_cli
command - then the directory where confd_cli was invoked is used.
tailf:global-no-duplicate
Specifies that only one instance with the name that is given as argument can be run at any time in
the system.

154

Actions

tailf:interrupt
Specifies which signal to use to interrupt the executable when the action invocation is interrupted.
The input parameters are passed to the executable as arguments (following those specified by
tailf:args) in the same general form as for a callback invocation. Each tag-value pair results in two
arguments, the first is the tag name as a string, the second is the value in string form. The special elements
used to indicate the start and end of a list entry or container and a typeless leaf (i.e. C_XMLBEGIN,
C_XMLEND, and C_XMLTAG in the C API) have the "values" __BEGIN, __END, and __LEAF. If the
action has output parameters, their values should be printed on standard output in the same form.
If the execution is successful, the executable should exit with code 0. Otherwise it may print error
information on standard output before exiting with a non-zero code. The error information can be either
a free-form string (corresponding to the confd_action_seterr() function for the callback) or
structured information corresponding to the NETCONF form described in the section called “EXTENDED
ERROR REPORTING” in the confd_lib_lib(3) manual page. It could for example print this on standard
output:
error-tag resource-denied
error-message "out of memory"

The error-tag element is required in this case.
In the CLI the action is not paginated by default and will only do so if it is piped to more.

joe@io> example_action | more

11.3.1. Example
Another action in intro/7-c_actions in the examples collection is implemented as a Perl script.
Here the data model defines a list of hosts, and an action that allows us to send ping requests to a host.
We specify the action in the data model like this:
list host {
key name;
leaf name {
type string;
}
tailf:action ping {
tailf:exec "./ping.pl" {
tailf:args "-c $(context) -p $(path)";
}
input {
leaf count {
type int32;
default "3";
}
}
output {
leaf header {
type string;
}
list response {
leaf data {
type string;
}

155

Actions

}
container statistics {
leaf packet {
type string;
}
leaf time {
type string;
}
}
}
}
}

The "-c $(context) -p $(path)" argument for the tailf:args statement has the effect that the context
(cli, netconf, etc) and the data model path will be passed to the script, followed by the count parameter
from the input statement in the data model. This parameter may have been given by the user or defaulted
according to the data model. Thus, if a host called "earth" exists in the configuration, and we use the JCLI and type this in operational mode:
request config host earth ping count 5

Then the script will be invoked as:
./ping.pl -c cli -p 'config host earth' count 5

The script starts by parsing these arguments, in particular picking up the last word of the -p value for
the host name (stored in the $host variable) and the argument for the count parameter (stored in the
$count variable):
while ($#ARGV >= 0) {
if ($ARGV[0] eq "-c") {
$context = $ARGV[1];
shift; shift;
} elsif ($ARGV[0] eq "-p") {
@path = split(' ', $ARGV[1]);
shift; shift;
} elsif ($ARGV[0] eq "count") {
$count = $ARGV[1];
shift; shift;
} else {
&fail("Unknown argument " . $ARGV[0]);
}
}
$host = $path[$#path];

In this example, the input parameters are very simple, just a single leaf. If we have lists or containers for
input in the data model, the script will receive them with __BEGIN and __END "values", as shown for
the output below.
Having collected the required parameters, and taking some OS dependencies into account, the script
proceeds to run the actual ping command, collecting the output (standard output and standard error) in
the $out variable, and checking the exit code. On a non-zero exit code (indicating failure), the fail
function will just print the output from ping on standard output and exit with code 1.
$ENV{'PATH'} = "/bin:/usr/bin:/sbin:/usr/sbin:" . $ENV{'PATH'};
if (`uname -s` eq "SunOS\n") {
$cmd = "ping -s $host 56 $count";
} else {
$cmd = "ping -c $count $host";

156

Actions

}
$out = `$cmd 2>&1`;
if ($? != 0) {
&fail($out);
}

If the execution of ping is successful, the script splits the output into lines and generates a reply according
to the output statement in the data model: each leaf is output with the leaf name followed by the value of
the leaf, and __BEGIN and __END "values" indicate the the start and end of each entry in the response
list, as well as the start and end of the statistics container:
@result = split('\n', $out);
print "header 'Invoked from " . $context . ": " . $result[0] . "'\n";
for ($i = 0; $i < $count; $i++) {
print "response __BEGIN data '" . $result[$i+1] . "' response __END\n";
}
$packets = $result[$#result-1];
$times = $result[$#result];
print "statistics __BEGIN\n";
print "packet '" . $packets . "' time '" . $times . "'\n";
print "statistics __END\n";
exit 0;

If we run the script interactively, using the command line above, we will get output that looks something
like this:

$ ./ping.pl -c cli -p 'config host earth' count 5
header 'Invoked from cli: PING earth.tail-f.com (192.168.1.42): 56 data bytes'
response __BEGIN data '64 bytes from 192.168.1.42: icmp_seq=0 ttl=64 time=0.187 ms' response
response __BEGIN data '64 bytes from 192.168.1.42: icmp_seq=1 ttl=64 time=0.150 ms' response
response __BEGIN data '64 bytes from 192.168.1.42: icmp_seq=2 ttl=64 time=0.208 ms' response
response __BEGIN data '64 bytes from 192.168.1.42: icmp_seq=3 ttl=64 time=0.205 ms' response
response __BEGIN data '64 bytes from 192.168.1.42: icmp_seq=4 ttl=64 time=0.204 ms' response
statistics __BEGIN
packet '5 packets transmitted, 5 packets received, 0.0% packet loss' time 'round-trip min/av
statistics __END

ConfD will parse this output and deliver the data to the requesting northbound agent. Since the values
include whitespace, they must be enclosed in quotes - either single quotes (') as in this example or double
quotes (") can be used. Arbitrary whitespace can be used to separate node names and values.

11.4. Related functionality
The action invocation mechanism is also used for some other related purposes:
• A NETCONF RPC (see Section 15.5, “Extending the NETCONF Server” in Chapter 15, The NETCONF
Server) can be specified to invoke a callback or an executable (where ConfD translates the XML),
via the tailf:actionpoint and tailf:exec statements, respectively. This is implemented via
invocation of the action() callback, or running of an executable, exactly as described above. (When
the tailf:raw-xml statement is used with tailf:exec, the argument and result passing described
above is not applicable.)
• The CLI can invoke "capi callbacks" for either complete CLI commands or command completion
functionality (see Chapter 16, The CLI agent). This is implemented via invocation of command() and
completion() callbacks, respectively, that are registered by the application in the same way as an

157

Actions

action() callback. See the section called “CONFD ACTIONS” in the confd_lib_dp(3) manual page
for the details.

158

Chapter 12. Notifications
12.1. ConfD Asynchronous Events
ConfD can deliver various classes of events to subscribing applications. The architecture is based on
notification sockets. The application(s) connect a notifications socket to ConfD. The application provides
a bit mask indicating which types of events the application is interested in. The application polls the socket
and invokes the API function confd_read_notification() whenever the socket is ready to read.
The API function populates a struct confd_notification structure.
The following is a list of the different asynchronous event classes that can be delivered from ConfD to the
application(s). See also the confd_lib_events(3) manual page. The program misc/notifications/
confd_notifications.c in the examples collection illustrates subscription and processing for all
these events, and can also be used standalone in a development environment to monitor ConfD events.
• CONFD_NOTIF_AUDIT - Audit events.
• CONFD_NOTIF_AUDIT_SYNC - Indicates that audit notifications (CONFD_NOTIF_AUDIT) must
be synced by the application.
• CONFD_NOTIF_DAEMON - Syslog events that also go to /confdConf/logs/confdLog.
• CONFD_NOTIF_NETCONF - Syslog events that also go to /confdConf/logs/netconfLog.
• CONFD_NOTIF_DEVEL - Syslog events that also go to /confdConf/logs/developerLog.
• CONFD_NOTIF_TAKEOVER_SYSLOG - Syslog control.
• CONFD_NOTIF_COMMIT_SIMPLE - Commit message.
• CONFD_NOTIF_COMMIT_DIFF - A complete diff compared to previous configuration.
• CONFD_NOTIF_COMMIT_FAILED - Possible data inconsistency event.
• CONFD_NOTIF_CONFIRMED_COMMIT - Events concerning confirmed commit processing.
• CONFD_NOTIF_COMMIT_PROGRESS - Events with commit progress information.
• CONFD_NOTIF_USER_SESSION - Whenever a user session is started or stopped.
• CONFD_NOTIF_HA_INFO - Changes in ConfD's perception of the cluster configuration.
• CONFD_NOTIF_HA_INFO_SYNC - Indicates that HA notifications (CONFD_NOTIF_HA_INFO)
must be synced by the application.
• CONFD_NOTIF_SUBAGENT_INFO - Subagent related events.
• CONFD_NOTIF_SNMPA - SNMP agent audit log.
• CONFD_NOTIF_FORWARD_INFO - Events related to forwarding (proxying) of northbound agents.
• CONFD_NOTIF_UPGRADE_EVENT - Events generated for in-service upgrade.
• CONFD_NOTIF_HEARTBEAT - Heartbeat events.

159

Notifications

• CONFD_NOTIF_HEALTH_CHECK - Health check events.
• CONFD_NOTIF_STREAM_EVENT - Notification stream events.

12.2. Audit Messages
Many applications need explicit control over where and in which format the various audit messages are
sent. By audit messages here we mean any message related to user login/logout/reconfig activity. The list
of different audit messages that are possible to receive can be found in the file confd_logsyms.h
In order to receive the audit message we must first connect a notifications socket.

Example 12.1. Creating a notification socket
confd_init("Foobar", stderr, debuglevel);
if ((notsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open notsocket\n");
inet_aton("127.0.0.1", &in);
addr.sin_addr.s_addr = in.s_addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(CONFD_PORT);
dflag = CONFD_NOTIF_AUDIT;
if (confd_notifications_connect(
notsock,
(struct sockaddr*)&addr,
sizeof (struct sockaddr_in), dflag) < 0 ) {
confd_fatal("Failed to confd_connect() to confd \n");
}

The dflags argument is bit mask indicating which classes of notifications messages we wish to receive
over the socket. It is possible to receive several different classes of notifications messages over the same
socket.
Once we have the socket setup, we add it to our pollset and invoke confd_read_notification()
once the socket is ready to read.

Example 12.2. reading the audit data
while (1) {
struct pollfd set[1];
struct confd_notification n;
set[0].fd = notsock;
set[0].events = POLLIN;
set[0].revents = 0;
if (poll(&set[0], 1, -1) < 0) {
perror("Poll failed:");
continue;
}
if (set[0].revents & POLLIN) {
if (confd_read_notification(notsock, &n) != CONFD_OK)
exit(1);
switch(n.type) {
case CONFD_NOTIF_AUDIT:
printf("audit: sym=%d, user=%s/%d %s\n",
n.n.audit.logno,

160

Notifications

n.n.audit.user, n.n.audit.usid, n.n.audit.msg);
break;
.......

The structure struct confd_notification is defined as:
enum confd_notification_type {
CONFD_NOTIF_AUDIT
CONFD_NOTIF_DAEMON
CONFD_NOTIF_TAKEOVER_SYSLOG
CONFD_NOTIF_COMMIT_SIMPLE
CONFD_NOTIF_COMMIT_DIFF
CONFD_NOTIF_USER_SESSION
CONFD_NOTIF_HA_INFO
CONFD_NOTIF_SUBAGENT_INFO
CONFD_NOTIF_COMMIT_FAILED
CONFD_NOTIF_SNMPA
CONFD_NOTIF_FORWARD_INFO
CONFD_NOTIF_NETCONF
CONFD_NOTIF_DEVEL
CONFD_NOTIF_HEARTBEAT
CONFD_NOTIF_CONFIRMED_COMMIT
CONFD_NOTIF_UPGRADE_EVENT
CONFD_NOTIF_COMMIT_PROGRESS
CONFD_NOTIF_AUDIT_SYNC
CONFD_NOTIF_HEALTH_CHECK
CONFD_NOTIF_STREAM_EVENT
CONFD_NOTIF_HA_INFO_SYNC
NCS_NOTIF_PACKAGE_RELOAD
NCS_NOTIF_CQ_PROGRESS
CONFD_NOTIF_REOPEN_LOGS
};

=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=

(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1
(1

<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<
<<

0),
1),
2),
3),
4),
5),
6),
7),
8),
9),
10),
11),
12),
13),
14),
15),
16),
17),
18),
19),
20),
21),
22),
23)

struct confd_notification {
enum confd_notification_type type;
union {
struct confd_audit_notification audit;
struct confd_syslog_notification syslog;
struct confd_commit_notification commit;
struct confd_commit_diff_notification commit_diff;
struct confd_user_sess_notification user_sess;
struct confd_ha_notification hnot;
struct confd_subagent_notification subagent;
struct confd_forward_notification forward;
struct confd_commit_failed_notification cfail;
struct confd_snmpa_notification snmpa;
struct confd_confirmed_commit_notification confirm;
struct confd_upgrade_notification upgrade;
struct confd_progress_notification progress;
struct confd_stream_notification stream;
struct ncs_cq_progress_notification cq_progress;
} n;
};

Where the field type indicates the type of the message. Depending on the type, one of the other union
structures is populated by the confd_read_notification() API function
In our case with audit messages, we get a struct confd_audit_notification structure populated.
struct confd_audit_notification {

161

Notifications

int logno;
/* number from confd_logsyms.h */
char user[MAXUSERNAMELEN];
char msg[BUFSIZ];
int usid;
/* session id (0 means - not applicable ) */
};

The logno is an integer which defines the event. All log and audit events generated by confd are
enumerated and documented in the include file confd_logsyms.h.
If we have indicated that we want to synchronize audit messages with ConfD, we must call
confd_sync_audit_notification() after receiving an audit message, to signal ConfD that it
can continue processing.

12.3. Syslog Messages
Some applications have explicit requirements not only where to send syslog messages (this can be easily
configured in confd.conf) but also how and on which format to send the syslog messages. By default,
ConfD will simply invoke the standard libc syslog() function.
It is possible to subscribe to ConfD syslog messages and also at the same time suppress ConfD's own
syslogging. To subscribe to syslog messages, the application needs to use one or more of the flags
CONFD_NOTIF_DAEMON, CONFD_NOTIF_NETCONF, and CONFD_NOTIF_DEVEL in the mask given
to confd_notifications_connect().
If the mask given to confd_notifications_connect() contains the flag
CONFD_NOTIF_TAKEOVER_SYSLOG, ConfD will not invoke the regular syslog() function. Thus
in this case, it is entirely up to the application to actually report the messages.
If all notifications subscribers that have requested the CONFD_NOTIF_TAKEOVER_SYSLOG feature
close their notifications sockets, ConfD will revert to the behavior of invoking libc syslog().
Similarly, when ConfD is starting, before any application processes has connected and requested the
CONFD_NOTIF_TAKEOVER_SYSLOG feature, ConfD will of course use the standard syslog()
functionality
When subscribing to syslog messages we receive a populated struct confd_syslog_notification structure:
struct confd_syslog_notification {
int prio;
/* from syslog.h */
int logno; /* number from confd_logsyms.h */
char msg[BUFSIZ];
};

The logno is an integer which defines the event. All syslog and audit events generated by confd are
enumerated and documented in the include file confd_logsyms.h.

12.4. Commit Events
There are two different types of commit events we can subscribe to. One really simple which just indicates
that a commit from a north bound agent has occurred. This is achieved by setting the subscription
bitmask to contain the flag: CONFD_NOTIF_COMMIT_SIMPLE. The message we receive contains a
struct confd_commit_notification structure:
struct confd_commit_notification {
enum confd_dbname database;

162

Notifications

int diff_available;
struct confd_user_info uinfo;
int flags;
};

This just provides information on which user committed to which database e.g. running or the candidate.
The other commit notification is considerably more complex and it provides information on exactly which
nodes were changed.
The flag value is CONFD_NOTIF_COMMIT_DIFF, and the structure we receive is:
struct confd_commit_diff_notification {
enum confd_dbname database;
struct confd_user_info uinfo;
struct confd_trans_ctx *tctx;
int flags;
char comment[MAX_COMMENT_LEN];
char label[MAX_LABEL_LEN];
};

The structure contains a transaction context which we can choose to use with maapi_attach()and thus
attach to the currently executing transaction. When the event is generated, this transaction has successfully
been committed by all data providers, but the commit operation has not completed and it is hanging, waiting
for the application to invoke confd_diff_notification_done(). The structure also includes the
"comment" and "label" given for the commit, if any (if not given, the comment and/or label elements
are zero-length strings).
maapi_attach() attaches a transaction context. We can then use that transaction context to read from
the transaction. The transaction has a list of nodes which constitute the configuration changes in the
transaction. We can traverse this list using the function maapi_diff_iterate() which will invoke
a user supplied function for each and every modification in the transaction.
The purpose of this feature is not to be able to check the commit diff. All such checking should be done
using the normal validation routines. The purpose is rather to be able to log diffs on a per commit basis.
Thus the first thing we need if we want to traverse the diff list is a function to be invoked for every diff
item. Our example here will just format the data and print to stdout.
static enum maapi_iter_ret iter(confd_hkeypath_t *kp,
enum maapi_iter_op op,
confd_value_t *oldv,
confd_value_t *v,
void *state)
{
char path[BUFSIZ];
char value[BUFSIZ];
char *opstr;
struct confd_cs_node *node;
confd_hkeypath_t *dkp;
int i;
confd_pp_kpath(path, sizeof(path), kp);
value[0] = 0;
switch (op) {
case MOP_CREATED:
opstr = "created";
break;
case MOP_DELETED:

163

Notifications

opstr = "deleted";
break;
case MOP_MODIFIED:
opstr = "modified";
break;
case MOP_VALUE_SET:
opstr = "value_set";
node = confd_find_cs_node(kp, kp->len);
confd_val2str(node->info.type, v, value, sizeof(value));
break;
case MOP_MOVED_AFTER:
if (v == NULL) {
opstr = "moved first";
} else {
opstr = "moved after";
/* create+print a hkeypath for the entry this one was moved after */
dkp = confd_hkeypath_dup(kp);
for (i = 0; v[i].type != C_NOEXISTS; i++) {
confd_free_value(&dkp->v[0][i]);
confd_value_dup_to(&v[i], &dkp->v[0][i]);
}
confd_pp_kpath(value, sizeof(value), dkp);
confd_free_hkeypath(dkp);
}
break;
case MOP_ATTR_SET:
if (v[1].type == C_NOEXISTS) {
opstr = "attr_del";
snprintf(value, sizeof(value), "%s", attr_str(&v[0]));
} else {
opstr = "attr_set";
i = snprintf(value, sizeof(value), "%s -> ", attr_str(&v[0]));
confd_pp_value(&value[i], sizeof(value) - i, &v[1]);
}
break;
}
printf ("ITER %s %s %s\n", path, opstr, value);
return ITER_RECURSE;
}

The iteration function must return an enum maapi_iter_ret indicating to ConfD what to continue to do.
We have the following possible return values:
• ITER_STOP - Stop. Do not invoke the iteration function any more for this transaction
• ITER_RECURSE - Iteration continues with all children of the modified node.
• ITER_CONTINUE - Iteration ignores the children of the node and continues with the node's sibling.
The iteration function is called for each modified node in the configuration. See the description
of maapi_diff_iterate() in confd_lib_maapi(3) for a detailed description of when the
different op values MOP_CREATED, MOP_DELETED, MOP_MODIFIED, MOP_VALUE_SET, and
MOP_MOVED_AFTER are used.
Finally we must have a function which is invoked whenever we receive a notification of type
CONFD_NOTIF_COMMIT_DIFF. The function must use the supplied transaction context and attach, and
when it is done traversing the diff it must call confd_diff_notification_done().
static void handle_diff_notif(struct confd_trans_ctx *tctx)

164

Notifications

{
/* first we need a maapi socket */
int maapi_socket;
if ((maapi_socket = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open socket\n");
if (maapi_connect(maapi_socket, (struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd \n");
/* no namespace needed for this */
OK(maapi_attach(maapi_socket, -1, tctx));
/* Now we can iterate through the currently hanging transaction */
/* and read out all the diffs */
OK(maapi_diff_iterate(maapi_socket, tctx->thandle, iter,
ITER_WANT_ATTR, NULL));
/* and finally call done to release data and let
/* the transaction finish */

*/

OK(confd_diff_notification_done(notif_socket, tctx));
close(maapi_socket);
}

12.5. Commit Failure Events
The CONFD_NOTIF_COMMIT_FAILED event is generated when a data provider fails in its commit
callback. ConfD executes a two-phase commit procedure towards all data providers when committing
transactions. When a provider fails in commit, the system is an unknown state. See confd_lib_maapi(3)
and the function maapi_get_running_db_status(). If the provider is "external", the name of the
failing daemon is provided. If the provider is another NETCONF agent, the IP address and port of that
agent is provided.

12.6. Confirmed Commit Events
When a a user has started a confirmed commit, when a confirming commit is issued, or when a confirmed
commit is aborted, a CONFD_NOTIF_CONFIRMED_COMMIT event is generated. The application
receives a struct confd_confirmed_commit_notification, which gives the specific action and user session
info for the committer. For a confirmed commit, the timeout value is also given.

12.7. Commit Progress Events
By subscribing to the CONFD_NOTIF_COMMIT_PROGRESS event, the application can receive the same
commit progress information that is reported when the commit | details CLI command is used. The
application receives a struct confd_progress_notification structure.

12.8. User Sessions
We can get notifications on user sessions and on user session events. A user session corresponds to an
actual user logging in to the system, for example a NETCONF manager
The struct confd_user_sess_notification structure is defined as:

165

Notifications

enum confd_user_sess_type {
CONFD_USER_SESS_START = 1,
/* a user session is started */
CONFD_USER_SESS_STOP = 2,
CONFD_USER_SESS_LOCK = 3,
/* a database is locked */
CONFD_USER_SESS_UNLOCK = 4,
CONFD_USER_SESS_START_TRANS = 5, /* a database transaction is started */
CONFD_USER_SESS_STOP_TRANS = 6
};
struct confd_user_sess_notification {
enum confd_user_sess_type type;
struct confd_user_info uinfo;
enum confd_dbname database;
};

This means that we can follow the progress of a user session, which databases are touched by the session
etc.

12.9. High Availability - Cluster Events
ConfD HA capabilities are described in Chapter 24, High Availability. This section describes the various
events that are asynchronously produced by ConfD when the cluster configuration is changed. These
changes may be induced explicitly by the application through invocation of the various HA related API
functions in libconfd or they may be induced by ConfD itself when the sockets between the HA nodes
get closed. It is vital that the High-Availability-Framework (HAFW) subscribes to these messages and
acts accordingly.
The struct confd_notification structure received by confd_read_notification() will populate the
hnot field with a struct confd_ha_notification. This in its turn is yet another union structure with a type
field.
struct confd_ha_notification {
enum confd_ha_info_type type;
/* additional info for various info types
union {
int nomaster;
/*
struct confd_ha_node slave_died;
/*
struct confd_ha_node slave_arrived;/*
int cdb_initialized_by_copy;
/*
int beslave_result;
/*
} data;
};

*/
CONFD_HA_INFO_NOMASTER */
CONFD_HA_INFO_SLAVE_DIED */
CONFD_HA_INFO_SLAVE_ARRIVED*/
CONFD_HA_INFO_SLAVE_INITIALIZED */
CONFD_HA_INFO_BESLAVE_RESULT */

We start with a listing of types of the different HA related events that ConfD can send to the subscribing
application. The enum is defined as:
enum confd_ha_info_type {
CONFD_HA_INFO_NOMASTER
CONFD_HA_INFO_SLAVE_DIED
CONFD_HA_INFO_SLAVE_ARRIVED
CONFD_HA_INFO_SLAVE_INITIALIZED
CONFD_HA_INFO_IS_MASTER
CONFD_HA_INFO_IS_NONE
CONFD_HA_INFO_BESLAVE_RESULT
};

=
=
=
=
=
=
=

1,
2,
3,
4,
5,
6,
7

/*
/*
/*
/*
/*
/*
/*

we have no master */
a slave disappeared */
a slave arrived to us */
CDB is initialized */
we are now master */
we are now none */
result of async beslave() */

Each of the different informational messages has additional data associated to it.

166

Notifications

• CONFD_HA_INFO_NOMASTER A node (which is a slave node) has lost contact with the master and is
now in HA state CONFD_HA_STATE_NONE. Only sent on the slave node.
Whenever we receive this message the nomaster field is populated. This is either the integer
CONFD_ERR_HA_CLOSED if the slave lost contact with master due to the socket getting closed or the
integer CONFD_ERR_HA_NOTICK if the slave has not received any live ticks from the master.
• CONFD_HA_INFO_SLAVE_DIED A master node lost contact with a slave node. Only sent on the
master node. The field slave_died is populated with a struct confd_ha_node indicating which
particular slave died.
• CONFD_HA_INFO_SLAVE_ARRIVED A master node was connected to by a slave node.
Authentication was ok and the slave is initializing its CDB database. Only sent at the master node. The
field slave_arrived is populated with a struct confd_ha_node indicating which slave arrived.
• CONFD_HA_INFO_SLAVE_INITIALIZED A slave node has just finished its initialization and
synchronization of the database. The slave is now fully operational. Only sent at slave nodes. The field
cdb_initialized_by_copy is set to 1 if ConfD concluded that the entire CDB database has to
be copied and 0 if a copy was avoided.
• CONFD_HA_INFO_IS_MASTER The node has been successfully elevated to master. This is only sent
at the master node, i.e. the node that just became master.
• CONFD_HA_INFO_IS_NONE The node has been set to NONE mode.
• CONFD_HA_INFO_BESLAVE_RESULT If we use asynchronous invocation of the
confd_ha_beslave() function, i.e. with the parameter waitreply set to 0, this message is
sent when the operation has completed. The field beslave_result is set to indicate the result
which would have been returned by a synchronous invocation of confd_ha_beslave(). Thus if
beslave_result is 0, the node has successfully become a slave, otherwise beslave_result
is one of the confd_errno values that can be returned by synchronous invocation of
confd_ha_beslave().
If we have indicated that we want to synchronize HA messages with ConfD, we must call
confd_sync_ha_notification() after receiving a HA message, to signal ConfD that it can
continue processing.

12.10. Subagent Events
The subagent mechanism is described in Chapter 26, Subagents and Proxies. This section describes the
related events which ConfD generates when acting as a master agent.
When the notification type is CONFD_NOTIF_SUBAGENT_INFO, the struct confd_notification structure
received by confd_read_notification() will populate the subagent field with a struct
confd_subagent_notification.
struct confd_subagent_notification {
enum confd_subagent_info_type type;
char name[MAXAGENTNAMELEN];
};

The
type
field
is
one
CONFD_SUBAGENT_INFO_DOWN.

of

the

values

CONFD_SUBAGENT_INFO_UP

or

At first, each subagent is marked as being down. When ConfD successfully communicates with a subagent,
it is marked as up, and a corresponding event is generated. A down event is generated only if ConfD tries

167

Notifications

to communicate with a subagent, but fails. Thus, if a subagent closes an idle connection to the master
agent, it is not marked as down.

12.11. SNMP Agent Audit Log
The SNMP agent log is activated through the /confdCfg/logs/snmpLog element in the
confd.conf configuration file.
The SNMP audit log messages can also be received and processed by an external C program over a
notification socket. The application receives a struct confd_snmpa_notification structure. The structure
contains a series of fields describing the sent or received SNMP PDU. It also contains a list of all varbinds
in the PDU.
Each varbind contains a confd_value_t with the string representation of the SNMP value. Thus the type
of the value in a varbind is always C_BUF. See confd_events.h include file for the details of the
received structure.
The following code exemplifies how we write a program which establishes a notification socket and
subscribes to all SNMP PDUs in and out of the system.
We start off with some auxiliary function to format the PDU type and the type of a "varbind"
char *vb_type(struct confd_snmp_varbind *vb) {
switch (vb->vartype) {
case CONFD_SNMP_NULL: return "NULL";
case CONFD_SNMP_INTEGER: return "INTEGER";
case CONFD_SNMP_Interger32: return "Integer32";
case CONFD_SNMP_OCTET_STRING: return "OCTET STRING";
case CONFD_SNMP_OBJECT_IDENTIFIER: return "OBJECT IDENTIFIER";
case CONFD_SNMP_IpAddress: return "IpAddress";
case CONFD_SNMP_Counter32: return "Counter32";
case CONFD_SNMP_TimeTicks: return "TimeTicks";
case CONFD_SNMP_Opaque: return "Opaque";
case CONFD_SNMP_Counter64: return "Counter64";
case CONFD_SNMP_Unsigned32: return "Unsigned32";
}
return "";
}
char *pdutype(struct confd_snmpa_notification *snmp) {
switch (snmp->pdu_type) {
case CONFD_SNMPA_PDU_V1TRAP: return("V1TRAP");
case CONFD_SNMPA_PDU_V2TRAP: return("V2TRAP");
case CONFD_SNMPA_PDU_INFORM: return("INFORM");
case CONFD_SNMPA_PDU_GET_RESPONSE: return("GET_RESPONSE");
case CONFD_SNMPA_PDU_GET_REQUEST: return("GET_REQUEST");
case CONFD_SNMPA_PDU_GET_NEXT_REQUEST: return("GET_NEXT_REQUEST");
case CONFD_SNMPA_PDU_REPORT: return("REPORT");
case CONFD_SNMPA_PDU_GET_BULK_REQUEST: return("GET_BULK_REQUEST");
case CONFD_SNMPA_PDU_SET_REQUEST: return("SET_REQUEST");
default: return "";
}
}

Following that we show the code which invokes confd_read_notification() and reads a C
structure of the type struct confd_snmpa_notification

168

Notifications

The structure contains the type of the PDU, various other fields and also the complete SNMP "varbind"
lists in the PDU. The code prints the PDU type and then loops through all the varbinds and prints the
value of each varbind.
if (confd_read_notification(notsock, &n) != CONFD_OK)
exit(1);
switch(n.type) {
case CONFD_NOTIF_SNMPA: {
int i,j;
char buf[BUFSIZ];
buf[0] = 0;
char *ptr = &buf[0];
struct confd_snmpa_notification *snmp = &n.n.snmpa;
ptr += sprintf(ptr, "%s ", pdutype(snmp));
ptr += sprintf(ptr,"Id = %d ", snmp->request_id);
struct confd_ip *ip = &(snmp->ip);
ptr += sprintf(ptr, " %s:%d ",
inet_ntoa(ip->ip.v4),
snmp->port);
if ((snmp->error_status !=0 || snmp->error_index != 0)) {
ptr += sprintf(ptr, "ErrIx = %d ", snmp->error_index);
}
else if (snmp->pdu_type == CONFD_SNMPA_PDU_V1TRAP) {
ptr += sprintf(ptr,"Generic=%d Specific=%d",
snmp->v1_trap->generic_trap,
snmp->v1_trap->specific_trap);
struct confd_snmp_oid *enterp = &snmp->v1_trap->enterprise;
ptr += sprintf(ptr, " Enterprise=");
for(i=0; i < enterp->len; i++) {
ptr += sprintf(ptr,".%d", enterp->oid[i]);
}
}
for (i=0; i < snmp->num_variables; i++) {
struct confd_snmp_varbind *vb = &snmp->vb[i];
ptr += sprintf(ptr,"\n
");
switch (vb->type) {
case CONFD_SNMP_VARIABLE:
ptr += sprintf(ptr, " %s ", vb_type(vb));
ptr += sprintf(ptr,"%s=", vb->var.name);
break;
case CONFD_SNMP_OID:
ptr += sprintf(ptr, " %s ", vb_type(vb));
for (j=0; j < vb->var.oid.len; j++) {
ptr += sprintf(ptr,"%d", vb->var.oid.oid[j]);
if (j != vb->var.oid.len-1)
ptr += sprintf(ptr,".");
}
break;
case CONFD_SNMP_COL_ROW:
ptr += sprintf(ptr, " %s ", vb_type(vb));
ptr += sprintf(ptr, "%s", vb->var.cr.column);
for(j=0; jvar.cr.rowindex.len; j++) {
ptr += sprintf(ptr,".%d",
vb->var.cr.rowindex.oid[j]);
}
break;
}
if (vb->val.type == C_BUF) {
char buf2[BUFSIZ];

169

Notifications

confd_pp_value(buf2, BUFSIZ, &vb->val);
ptr += sprintf(ptr, "=%s", buf2);
}
}
printf("%s\n\n", buf);
confd_free_notification(&n);
}

12.12. Forwarding Events
ConfD can forward (proxy) connections from northbound agents. When forwarding starts, ends,
or fails, a CONFD_NOTIF_FORWARD_INFO event is generated. The application receives a struct
confd_forward_notification structure which gives the type of forwarding event, the name of the target for
the forwarding, and user session information for the user that requested the forwarding.

12.13. In-service Upgrade Events
During in-service upgrade, the CONFD_NOTIF_UPGRADE_EVENT event is generated with different
values for the enum confd_upgrade_event_type event. The events correspond to the different phases
of the upgrade, see Chapter 13, In-service Data Model Upgrade and confd_lib_maapi(3) for a detailed
description.

12.14. Heartbeat and Health Check Events
The CONFD_NOTIF_HEARTBEAT and CONFD_NOTIF_HEALTH_CHECK events can be used by
applications that wish to monitor the health and liveness of ConfD itself. See confd_lib_events(3) for more
details about this.

12.15. Notification stream Events
The CONFD_NOTIF_STREAM_EVENT event is generated for a notification stream, i.e. event
notifications sent by an application as described in the section called “NOTIFICATION STREAMS” of
confd_lib_dp(3). See confd_lib_events(3) for more details about this.

170

Chapter 13. In-service Data Model
Upgrade
13.1. Introduction
When we want to change the data model used by ConfD, the simplest method is to stop and restart ConfD
with the new .fxs files in place. CDB will then detect the change and perform an upgrade, automatically
or assisted by external programs, as described in Chapter 5, CDB - The ConfD XML Database.
If it is necessary that ConfD keeps running throughout the data model change, we can instead control the
upgrade from an external program using a set of MAAPI functions, as described in this chapter. The CDB
upgrade will be performed in this case too of course, and all the techniques described in the CDB chapter
are applicable here too. But in addition, this procedure requires careful synchronization between different
ConfD components, e.g. transactions may not span the data model change, and all components must update
any related data, while still being able to revert to the original data model in case problems are detected.
The following four sections describe the phases and corresponding MAAPI function calls that
comprise this upgrade procedure, and the steps that must be taken by the program controlling
the procedure. A complete example showing the procedure can be found in examples.confd/
in_service_upgrade/simple in the bundled examples collection. The code excerpts and
description refer to this example. See also the confd_lib_maapi(3) manual page for the definitions of the
MAAPI functions.

13.2. Preparing for the Upgrade
All the new .fxs files, clispecs, MIBs, etc, as well as the docroot tree for the Web UI (if used), that are
to be used after the upgrade, must be installed separately from the current ones, and the current ones may
not be removed until the upgrade has completed successfully. A good way to organize this is to have the
references to the installation directories in confd.conf use a symbolic link. This way we can switch the
on-disk data to the new version by simply changing the link, without the need for complex modification
of confd.conf. Files that are unchanged between the two versions should be duplicated, or possibly
(hard-)linked if disk space is limited.
In the example, we use two directories pkg/v1 and pkg/v2 for the old and new versions, respectively,
and a symlink pkg/current that points to the currently used version. In confd.conf both /
confdConfig/loadPath/dir and /confdConfig/webui/docroot are then given with the
use of the symlink. Multiple loadPath directories are of course also possible, by having subdirectories
below v1 and/or v2.
For MIB (.bin) files other than the ones built-in to ConfD, confd.conf offers two possibilities: we
can either specify the actual file names with /confdConfig/snmpAgent/mibs/file elements, or
use /confdConfig/snmpAgent/mibs/fromLoadPath to tell ConfD to load these files from the
directories given via /confdConfig/loadPath. To have the symlink scheme work for the MIB files
on upgrade, we need to use the latter alternative, and only specify built-in MIBs via /confdConfig/
snmpAgent/mibs/file.
The upgrade.c program in the example controls the upgrade procedure. It can be run either standalone
or via a osCommand specification in the clispec. In both cases it must connect a MAAPI socket and
associate it with a user session. When running from the CLI, it must use the user session id provided by
the environment variable CONFD_MAAPI_USID for this, otherwise it can start a new user session:

171

In-service Data Model Upgrade

static int set_usess(int sock)
{
char *user = "admin";
const char *groups[] = {"admin"};
char *context = "system";
struct confd_ip ip;
/* must use usid from CLI to be allowed to run across upgrade */
if ((usid_env = getenv("CONFD_MAAPI_USID")) != NULL) {
return maapi_set_user_session(sock, atoi(usid_env));
} else {
ip.af = AF_INET;
inet_pton(AF_INET, "127.0.0.1", &ip.ip.v4);
return maapi_start_user_session(sock, user, context, groups, 1,
&ip, CONFD_PROTO_TCP);
}
}

Applications connected to ConfD, e.g. data providers and CDB subscribers, are not directly affected by
the upgrade procedure. If they need to take some action due to the upgrade, they should be subscribed
to CONFD_NOTIF_UPGRADE_EVENT event notifications (see Chapter 12, Notifications), and will
then be notified of the different phases of the upgrade. If nothing else, most applications should call
confd_load_schemas() (see confd_lib_lib(3)) when an upgrade has completed, in order to update
the in-memory representation of the data model. The cdb_subscriber.c program in the example
shows how this can be done.

13.3. Initializing the Upgrade
After having set up the MAAPI socket, the first step in the actual upgrade procedure is to call the
maapi_init_upgrade() function. Its purpose is to bring ConfD into "upgrade mode", where no
transactions are running, and the northbound agents have entered a state that does not allow new
transactions to be started.
progress("Initializing upgrade...\n");
phase = "Init";
/* run notifier in separate process
- maapi_init_upgrade() blocks */
notifier = run_notifier(timeout, force);
OK(maapi_init_upgrade(maapisock, timeout,
force ? MAAPI_UPGRADE_KILL_ON_TIMEOUT : 0));
if (notifier != -1)
kill(notifier, SIGTERM);
notifier = -1;
progress("Init OK\n");
maapi_prio_message(maapisock, "all",
"\n>>> System upgrade in progress...\n");

If users have sessions in configure mode when this function is called, they are given the opportunity to exit
from configure mode voluntarily. The function call will block until all transactions have been terminated
(although not longer than specified by the timeout). For this reason, we fork() a process that periodically
sends out messages to all users, informing them of the imminent upgrade.
If any transactions remain when the timeout expires, maapi_init_upgrade() will fail
with confd_errno CONFD_ERR_TIMEOUT, unless the MAAPI_UPGRADE_KILL_ON_TIMEOUT
flag was used. If upgrade.c was given the -f (force) option, it will pass this flag to
maapi_init_upgrade(), and any remaining transactions will be forcibly terminated instead.

172

In-service Data Model Upgrade

When maapi_init_upgrade() is called, a CONFD_UPGRADE_INIT_STARTED event notification
is sent, and when it completes successfully, a CONFD_UPGRADE_INIT_SUCCEEDED event notification
is sent.

Note
If the function fails, i.e. it does not return CONFD_OK, ConfD will automatically abort the
upgrade, reverting to the pre-upgrade state, and send a CONFD_UPGRADE_ABORTED event
notification. This is true also for the functions described in in the next two sections.

13.4. Performing the Upgrade
When maapi_init_upgrade() has completed successfully, the next step is to call
maapi_perform_upgrade(). This tells ConfD to load the new .fxs files etc, and we must pass it
a list of directories to load these files from. These are the directories that will become the new loadPath
directories once the upgrade is complete. These directories will also be searched for CDB "init files" (see
Section 5.8, “Loading initial data into CDB”), corresponding to the /confdConfig/cdb/initPath
directories that can be specified in confd.conf.
progress("Performing upgrade...\n");
phase = "Perform";
/* set up new loadpath directory */
snprintf(buf, sizeof(buf), PKG_DIR "/%s", version);
load_dir[0] = &buf[0];
OK(maapi_perform_upgrade(maapisock, &load_dir[0], ndirs));
progress("Perform OK\n");

At this point confd.conf and hence the current symlink must still point to the current version, and
thus we pass the new directory "explicitly" to the function as "./pkg/v2". In this example ConfD was
also started with the --addloadpath option specifying an additional loadPath directory. The contents
of this directory ($CONFD_DIR/etc/confd) does not change in the upgrade, but we must pass the same
directory to maapi_perform_upgrade() too - the files found in the given directories will completely
replace what ConfD is currently using.
A number of different problems may be detected during the loading of the new files, e.g. .fxs files may
have a version that is incompatible with the ConfD version, or they may reference namespaces that can
not be found. These problems will make maapi_perform_upgrade() fail with confd_errno
CONFD_ERR_BAD_CONFIG, and confd_lasterr() giving information about the details of the
problem. If the loading is successful, CDB will start its special upgrade transaction, and perform any
automatic upgrade operations that are needed, before maapi_perform_upgrade() returns.
When maapi_perform_upgrade() completes successfully, a CONFD_UPGRADE_PERFORMED
event notification is sent.

13.5. Committing the Upgrade
When
maapi_perform_upgrade()
has
completed
successfully,
we
must
call
maapi_commit_upgrade() to tell ConfD to make the upgrade permanent. This will also tell CDB to
commit its upgrade transaction, and we may need to take some actions for this before the call:
• If the upgrade requires that an external program modifies some CDB data, it must be done at this point,
using maapi_attach_init() as described in the CDB chapter.

173

In-service Data Model Upgrade

• If the upgrade includes new validation points, or the validation logic for existing validation points
has changed, the new validators must connect to ConfD and register for their validation points before
maapi_commit_upgrade() is called.
In the example, all the changes can be handled by the automatic CDB upgrade, and we just proceed with
the call:
progress("Committing upgrade...\n");
phase = "Commit";
OK(maapi_commit_upgrade(maapisock));
relink(version);
progress("Commit OK\n");
maapi_prio_message(maapisock, "all",
">>> System upgrade has completed successfully.\n");

maapi_commit_upgrade() may fail if the upgraded data does not pass validation, and the errors
returned in this case are the same as for e.g. maapi_apply_trans(). Since this will also make ConfD
automatically revert to the pre-upgrade state, we must not change the on-disk data to reflect the upgrade
until maapi_commit_upgrade() has succeeded. In the code above, the relink() call changes the
symlink to point to the new version in an atomic manner.
When maapi_commit_upgrade() completes successfully, a CONFD_UPGRADE_COMMITED event
notification is sent.

13.6. Aborting the Upgrade
We can abort the upgrade at any point before the maapi_commit_upgrade() call by calling
maapi_abort_upgrade(). However as noted above, this should not be done when one of the other
functions fails, since ConfD aborts the upgrade automatically in those cases.
When maapi_abort_upgrade() aborts an upgrade, a CONFD_UPGRADE_ABORTED event
notification is sent.

13.7. Upgrade and HA
When we use the ConfD High Availability functionality, it is critical that all nodes in the HA cluster agree
on the data model used. For this reason we can not do in-service upgrade on a ConfD instance that is part
of a HA cluster. A ConfD node in HA state SLAVE or MASTER executing maapi_init_upgrade()
will fail with confd_errno CONFD_ERR_HA_WITH_UPGRADE. Conversely, when an in-service
upgrade is in progress, calling confd_ha_beslave() will also result in this error, and connections
from slaves will be rejected.
Thus in-service upgrade is only possible while in HA-state NONE.
To do the in-service upgrade on a HA cluster, we must thus use "rolling upgrade":
1. Disconnect one of the slaves from the cluster by calling confd_ha_benone().
2. Upgrade the disconnected slave as described above.
3. Tell the upgraded slave to become master by calling confd_ha_bemaster().
4. Tell the old master to not be master by calling confd_ha_benone()

174

In-service Data Model Upgrade

5. Upgrade the remaining nodes in the cluster one by one, telling each to connect as slave to the upgraded
master by calling confd_ha_beslave() when the upgrade is done.
Alternatively, since the HA configuration should be able to handle that a node is stopped and restarted
without service interruption, we may simply use the upgrade method described in the CDB chapter for
the "rolling upgrade".

175

Chapter 14. The AAA infrastructure
14.1. The problem
This chapter describes how to use ConfD's built-in authentication and authorization mechanisms. Users
log into ConfD through the CLI, NETCONF, RESTCONF, SNMP, or via the Web UI. In either case, users
need to be authenticated. That is, a user needs to present credentials, such as a password or a public key
in order to gain access. As an alternative for RESTCONF, users can be authenticated via token validation.
Once a user is authenticated, all operations performed by that user need to be authorized. That is, certain
users may be allowed to perform certain tasks, whereas others are not. This is called authorization. We
differentiate between authorization of commands and authorization of data access.

14.2. Structure - data models
The ConfD daemon manages device configuration including AAA information. In fact, ConfD both
manages AAA information and uses it. The AAA information describes which users may login, what
passwords they have and what they are allowed to do.
This is solved in ConfD by requiring a data model to be both loaded and populated with data.
ConfD uses the YANG module tailf-aaa.yang for authentication, while ietf-netconfacm.yang (NACM, RFC 6536) as augmented by tailf-acm.yang is used for group assignment and
authorization. For backwards compatibility, it is alternatively possible to use the older revision 2011-09-22
of tailf-aaa.yang for all of authentication, group assignment, and authorization. This legacy version
of tailf-aaa can be found in the $CONFD_DIR/src/confd/aaa directory, but its usage is not further
described here. Detailed information about this can be found in versions of this document for ConfD-5.3
and earlier.

14.2.1. Data model contents
The NACM data model is targeted specifically towards access control for NETCONF operations, and thus
lacks some functionality that is needed in ConfD, in particular support for authorization of CLI commands
and the possibility to specify the "context" (NETCONF/CLI/etc) that a given authorization rule should
apply to. This functionality is modeled by augmentation of the NACM model, as defined in the tailfacm.yang YANG module.
The ietf-netconf-acm.yang and tailf-acm.yang modules can be found in $CONFD_DIR/
src/confd/yang directory in the release, while tailf-aaa.yang can be found in the
$CONFD_DIR/src/confd/aaa directory.
The complete AAA data model defines a set of users, a set of groups and a set of rules. The data model
must be populated with data that is subsequently used by by ConfD itself when it authenticates users and
authorizes user data access. These YANG modules work exactly like all other fxs files loaded into the
system with the exception that ConfD itself uses them. The data belongs to the application, but ConfD
itself is the user of the data.
Since ConfD requires a data model for the AAA information for its operation, it will report an error and
fail to start if these data models can not be found.

176

The AAA infrastructure

14.3. AAA related items in confd.conf
ConfD itself is configured through a configuration file - confd.conf . In that file we have the following
items related to authentication and authorization:
/confdConfig/aaa/
sshServerKeyDir

If SSH termination is enabled for NETCONF or the CLI, the ConfD
built-in SSH server needs to have server keys. These keys are
generated by the ConfD install script and by default end up in
$CONFD_DIR/etc/confd/ssh .
It is also possible to use OpenSSH to terminate NETCONF or the
CLI. If OpenSSH is used to terminate SSH traffic, the SSH keys
are not necessary.

/confdConfig/aaa/
sshPubkeyAuthentication

If SSH termination is enabled for NETCONF or the CLI, this item
controls how the ConfD SSH daemon locates the user keys for
public key authentication. See Section 14.4.1, “Public Key Login”
for the details.

/confdConfig/aaa/
localAuthentication/
enabled

The term "local user" refers to a user stored under /aaa/
authentication/users. The alternative is a user unknown
to ConfD, typically authenticated by PAM.
By default, ConfD first checks local users before trying PAM or
external authentication.
Local authentication is practical in test environments. It is also
useful when we want to have one set of users that are allowed
to login to the host with normal shell access and another set of
users that are only allowed to access the system using the normal
encrypted, fully authenticated, northbound interfaces of ConfD.
If we always authenticate users through PAM it may make sense to
set this configurable to false. If we disable local authentication
it implicitly means that we must use either PAM authentication or
"external authentication". It also means that we can leave the entire
data trees under /aaa/authentication/users and, in the
case of "external auth" also /nacm/groups (for NACM) or /
aaa/authentication/groups (for legacy tailf-aaa) empty.

/confdConfig/aaa/pam

ConfD can authenticate users using PAM (Pluggable
Authentication Modules). PAM is an integral part of most Unixlike systems.
PAM is a complicated - albeit powerful - subsystem. It may be
easier to have all users stored locally on the host, However if we
want to store users in a central location, PAM can be used to
access the remote information. PAM can be configured to perform
most login scenarios including RADIUS and LDAP. One major
drawback with PAM authentication is that there is no easy way to
extract the group information from PAM. PAM authenticates users,
it does not also assign a user to a set of groups.
PAM authentication is thoroughly described later in this chapter.
177

The AAA infrastructure

/confdConfig/aaa/
defaultGroup

If this configuration parameter is defined and if the group of a user
cannot be determined, a logged in user ends up in the given default
group.

/confdConfig/aaa/
aaaBridge

This key will be described in the Section 14.9, “Populating AAA
using external data” section.

/confdConfig/aaa/
externalAuthentication

ConfD can authenticate users using an external executable.
This is further described later in the Section 14.4.4, “External
authentication” section.

/confdConfig/aaa/
externalValidation

ConfD can authenticate users by validation of tokens using
an external executable. This is further described later in the
Section 14.4.5, “External token validation” section. The difference
is that a token, instead of a username and password, is input and a
username and, optionally, a token is output. This is currently only
supported for RESTCONF.

/confdConfig/aaa/
authenticationCallback/
enabled

If this is set to "true", ConfD will, as the last step of
every authentication attempt, invoke an application callback. The
callback can reject an otherwise successful authentication. See
the section called “AUTHENTICATION CALLBACK” in the
confd_lib_dp(3) manual page for the details about this.

/confdConfig/aaa/
authorization/callback/
enabled

If this is set to "true", ConfD will invoke application callbacks
for authorization. The callbacks can partially or completely
replace the logic described in Section 14.6, “Authorization”. See
the section called “AUTHORIZATION CALLBACKS” in the
confd_lib_dp(3) manual page for the details about this.

14.4. Authentication
Depending on northbound management protocol, when a user session is created in ConfD, it may or
may not be authenticated. If the session is not yet authenticated, ConfD's AAA subsystem is used to
perform authentication and authorization, as described below. If the session already has been authenticated,
ConfD's AAA assigns groups to the user as described in Section 14.5, “Group Membership”, and performs
authorization, as described in Section 14.6, “Authorization”.
The authentication part of the data model can be found in tailf-aaa.yang:
container authentication {
tailf:info "User management";
container users {
tailf:info "List of local users";
list user {
key name;
leaf name {
type string;
tailf:info "Login name of the user";
}
leaf uid {
type int32;
mandatory true;
tailf:info "User Identifier";
}

178

The AAA infrastructure

leaf gid {
type int32;
mandatory true;
tailf:info "Group Identifier";
}
leaf password {
type passwdStr;
mandatory true;
}
leaf ssh_keydir {
type string;
mandatory true;
tailf:info "Absolute path to directory where user's ssh keys
may be found";
}
leaf homedir {
type string;
mandatory true;
tailf:info "Absolute path to user's home directory";
}
}
}
}

AAA authentication is used in the following cases:
• When the built-in SSH server is used for NETCONF and CLI sessions.
• For Web UI sessions and REST access.
• When the function maapi_authenticate() is used.
The different authentication mechanisms that may be used in these cases are described below. Regardless
of which mechanism that is used, ConfD can optionally invoke an application callback as the last
step of the authentication process, see the section called “AUTHENTICATION CALLBACK” in
confd_lib_dp(3). The callback is not used for the actual authentication, but it can reject an otherwise
successful authentication.
ConfD's AAA authentication is not used in the following cases:
• When NETCONF uses an external SSH daemon, such as OpenSSH.
In this case, the NETCONF session is initiated using the program netconf-subsys, as described in
Section 15.3, “NETCONF Transport Protocols”.
• When NETCONF uses TCP, as described in Section 15.3, “NETCONF Transport Protocols”, e.g.
through the command netconf-console.
• When the CLI uses an external SSH daemon, such as OpenSSH, or a telnet daemon.
In this case, the CLI session is initiated through the command confd_cli . An important special case
here is when a user has logged in to the host and invokes the command confd_cli from the shell.
• When SNMP is used. SNMP has its own authentication mechanisms. See Section 17.5.2, “USM and
VACM and ConfD AAA” .
• When the function maapi_start_user_session() is used without a preceding call of
maapi_authenticate().

179

The AAA infrastructure

14.4.1. Public Key Login
When a user logs in over NETCONF or the CLI using the built-in SSH server, with public key login, the
procedure is as follows.
The user presents a username in accordance with the SSH protocol. The SSH server consults the
settings for /confdConfig/aaa/sshPubkeyAuthentication and /confdConfig/aaa/
localAuthentication/enabled .
1. If
sshPubkeyAuthentication
is set to local, and the SSH keys in /aaa/
authentication/users/user{$USER}/ssh_keydir match the keys presented by the user,
authentication succeeds.
2. Otherwise, if sshPubkeyAuthentication is set to system, localAuthentication is
enabled, and the SSH keys in /aaa/authentication/users/user{$USER}/ssh_keydir
match the keys presented by the user, authentication succeeds.
3. Otherwise, if sshPubkeyAuthentication is set to system and the user /aaa/
authentication/users/user{$USER} does not exist, but the user does exist in the OS
password database, the keys in the user's $HOME/.ssh directory are checked. If these keys match the
keys presented by the user, authentication succeeds.
4. Otherwise, authentication fails.
In all cases the keys are expected to be stored in a file called authorized_keys (or
authorized_keys2 if authorized_keys does not exist), and in the native OpenSSH format
(i.e. as generated by the OpenSSH ssh-keygen command). If authentication succeeds, the user's group
membership is established as described in Section 14.5, “Group Membership”.
This is exactly the same procedure that is used by the OpenSSH server with the exception that the builtin SSH server also may locate the directory containing the public keys for a specific user by consulting
the /aaa/authentication/users tree.

Setting up Public Key Login
We need to provide a directory where SSH keys are kept for a specific user, and give the absolute path
to this directory for the /aaa/authentication/users/user/ssh_keydir leaf. If public key
login is not desired at all for a user, the value of the ssh_keydir leaf should be set to "", i.e. the empty
string. Similarly, if the directory does not contain any SSH keys, public key logins for that user will be
disabled.
The built-in SSH daemon supports DSA and RSA keys. To generate and enable RSA keys of size 4096
bits for, say, user "bob", the following steps are required.
On the client machine, as user "bob", generate a private/public key pair as:

# ssh-keygen -b 4096 -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/bob/.ssh/id_rsa):
Created directory '/home/bob/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/bob/.ssh/id_rsa.
Your public key has been saved in /home/bob/.ssh/id_rsa.pub.

180

The AAA infrastructure

The key fingerprint is:
ce:1b:63:0a:f9:d4:1d:04:7a:1d:98:0c:99:66:57:65 bob@buzz
# ls -lt ~/.ssh
total 8
-rw------- 1 bob users 3247 Apr 4 12:28 id_rsa
-rw-r--r-- 1 bob users 738 Apr 4 12:28 id_rsa.pub

Now we need to copy the public key to the target machine where the NETCONF or CLI SSH client runs.
Assume we have the following user entry:


bob
100
10
$1$feedbabe$nGlMYlZpQ0bzenyFOQI3L1
/var/system/users/bob/.ssh
/var/system/users/bob


We need to copy the newly generated file id_rsa.pub, which is the public key, to a file on the target
machine called /var/system/users/bob/.ssh/authorized_keys

14.4.2. Password Login
Password login is triggered in the following cases:
• When a user logs in over NETCONF or the CLI using the built in SSH server, with a password. The
user presents a username and a password in accordance with the SSH protocol.
• When a user logs in using the Web UI. The Web UI asks for a username and password.
• When the function maapi_authenticate() is used.
In this case, ConfD will by default try local authentication, PAM, and external authentication, in that
order, as described below. It is possible to change the order in which these are tried, by modifying the
confd.conf . parameter /confdConfig/aaa/authOrder . See confd.conf(5) for details.
1. If /aaa/authentication/users/user{$USER} exists and the presented password matches
the encrypted password in /aaa/authentication/users/user{$USER}/password the
user is authenticated.
2. If the password does not match or if the user does not exist in /aaa/authentication/users,
PAM login is attempted, if enabled. See ???? for details.
3. If all of the above fails and external authentication is enabled, the configured executable is invoked.
See ???? for details.
If authentication succeeds, the user's group membership is established as described in ????.

14.4.3. PAM
On operating systems supporting PAM, ConfD also supports PAM authentication. Using PAM
authentication with ConfD can be very convenient since it allows us to have the same set of users and
groups having access to ConfD as those that have access to the UNIX/Linux host itself.

181

The AAA infrastructure

If we use PAM, we do not have to have any users or any groups configured in the ConfD aaa namespace
at all. To configure PAM we typically need to do the following:
1. Remove all users and groups from the aaa initialization XML file.
2. Enable PAM in confd.conf by adding:


true
common-auth


to the aaa section in confd.conf . The service name specifies the PAM service, typically a file in the
directory /etc/pam.d, but may alternatively be an entry in a file /etc/pam.conf, depending on
OS and version. Thus it is possible to have a different login procedure to ConfD than to the host itself.
3. If pam is enabled and we want to use pam for login the system may have to run as root. This depends
on how pam is configured locally. However the default "system-auth" will typically require root since
the pam libraries then read /etc/shadow. If we don't want to run ConfD as root, the solution here is
to change owner of a helper program called $CONFD_DIR/lib/confd/lib/core/pam/priv/
epam and also set the setuid bit.

# cd $CONFD_DIR/lib/confd/lib/core/pam/priv/
# chown root:root epam
# chmod u+s epam

PAM is the recommended way to authenticate ConfD users.
As an example, say that we have user test in /etc/passwd, and furthermore:

# grep test /etc/group
operator:x:37:test
admin:x:1001:test

thus, the test user is part of the admin and the operator groups and logging in to ConfD as the test user,
through CLI ssh, Web UI, or netconf renders the following in the audit log.


in

to


28-Jan-2009::16:05:55.663 buzz confd[14658]: audit user: test/0 logged
over ssh from 127.0.0.1 with authmeth:password
28-Jan-2009::16:05:55.670 buzz confd[14658]: audit user: test/5 assigned
groups: operator,admin
28-Jan-2009::16:05:57.655 buzz confd[14658]: audit user: test/5 CLI 'exit'

Thus, the test user was found and authenticated from /etc/passwd, and the crucial group assignment
of the test user was done from /etc/group.
If we wish to be able to also manipulate the users, their passwords etc on the device we can write a private
YANG model for that data, store that data in CDB, setup a normal CDB subscriber for that data, and
182

The AAA infrastructure

finally when our private user data is manipulated, our CDB subscriber picks up the changes and changes
the contents of the relevant /etc files.

14.4.4. External authentication
A common situation is when we wish to have all authentication data stored remotely, not locally, for
example on a remote RADIUS or LDAP server. This remote authentication server typically not only stores
the users and their passwords, but also the group information.
If we wish to have not only the users, but also the group information stored on a remote server, the best
option for ConfD authentication is to use "external authentication".
If this feature is configured, ConfD will invoke the executable configured in /confdConfig/aaa/
externalAuthentication/executable in confd.conf , and pass the username and the clear
text password on stdin using the string notation: "[user;password;]\n".
For example if user "bob" attempts to login over SSH using the password "secret", and
external authentication is enabled, ConfD will invoke the configured executable and write
"[bob;secret;]\n" on the stdin stream for the executable.
The task of the executable is then to authenticate the user and also establish the username-to-groups
mapping.
For example the executable could be a RADIUS client which utilizes some proprietary vendor attributes to
retrieve the groups of the user from the RADIUS server. If authentication is successful, the program should
write "accept " followed by a space-separated list of groups the user is member of, and additional
information as described below. Again, assuming that Bob's password indeed was "secret", and that Bob is
member of the "admin" and the "lamers" groups, the program should write "accept admin lamers
$uid $gid $supplementary_gids $HOME\n" on its standard output and then exit.
Thus the format of the output from an "externalauth" program when authentication is successful should be:
"accept $groups $uid $gid $supplementary_gids $HOME\n"
Where
• $groups is a space separated list of the group names the user is a member of.
• $uid is the UNIX integer user id ConfD should use as default when executing commands for this user.
• $gid is the UNIX integer group id ConfD should use as default when executing commands for this user.
• $supplementary_gids is a (possibly empty) space separated list of additional UNIX group ids the
user is also a member of.
• $HOME is the directory which should be used as HOME for this user when ConfD executes commands
on behalf of this user.
It is further possible for the program to return a token on successful authentication, by using
"accept_token" instead of "accept":
"accept_token $groups $uid $gid $supplementary_gids $HOME $token\n"
Where $token is an arbitrary string. ConfD will then, for some northbound interfaces, include this token
in responses.

183

The AAA infrastructure

It is also possible for the program to return additional information on successful authentication, by using
"accept_info" instead of "accept":
"accept_info $groups $uid $gid $supplementary_gids $HOME $info\n"
Where $info is some arbitrary text. ConfD will then just append this text to the generated audit log
message (CONFD_EXT_LOGIN).
Yet another possibility is for the program to return a warning that the user's password is about to expire,
by using "accept_warning" instead of "accept":
"accept_warning $groups $uid $gid $supplementary_gids $HOME $warning\n"
Where $warning is an appropriate warning message. The message will be processed by ConfD according
to the setting of /confdConfig/aaa/expirationWarning in confd.conf .
There is also support for token variations of "accept_info" and "accept_warning" namely
"accept_token_info" and "accept_token_warning". Both "accept_token_info" and
"accept_token_warning" expects the external program to output exactly the same as described
above with the addition of a token after $HOME:
"accept_token_info $groups $uid $gid $supplementary_gids $HOME $token
$info\n"
"accept_token_warning $groups $uid $gid $supplementary_gids $HOME $token
$warning\n"
If authentication failed, the program should write "reject" or "abort", possibly followed by a
reason for the rejection, and a trailing newline. For example "reject Bad password\n" or just
"abort\n". The difference between "reject" and "abort" is that with "reject", ConfD will
try subsequent mechanisms configured for /confdConfig/aaa/authOrder in confd.conf (if
any), while with "abort", the authentication fails immediately. Thus "abort" can prevent subsequent
mechanisms from being tried, but when external authentication is the last mechanism (as in the default
order), it has the same effect as "reject".
When external authentication is used, the group list returned by the external program is prepended by any
possible group information stored locally under the /aaa tree. Hence when we use external authentication
it is indeed possible to have the entire /aaa/authentication tree empty. The group assignment
performed by the external program will still be valid and the relevant groups will be used by ConfD when
the authorization rules are checked.

14.4.5. External token validation
When username, password authentication is not feasible, authentication by token validation is possible.
Currently only RESTCONF supports this mode of authentication. It shares all properties of external
authentication, but instead of a username and password, it takes a token as input. The output is also almost
the same, the only difference is that it is also expected to output a username.
If this feature is configured, ConfD will invoke the executable configured in /confdConfig/aaa/
externalValidation/executable in confd.conf , and pass the token on stdin using the
string notation: "[token;]\n".
For example if user "bob" attempts to login over RESTCONF using the token "topsecret", and external
validation is enabled, ConfD will invoke the configured executable and write "[topsecret;]\n" on
the stdin stream for the executable.

184

The AAA infrastructure

The task of the executable is then to validate the token, thereby authenticating the user and also establish
the username and username-to-groups mapping.
For example the executable could be a FUSION client which utilizes some proprietary vendor attributes
to retrieve the username and groups of the user from the FUSION server. If token validation is successful,
the program should write "accept " followed by a space-separated list of groups the user is member of,
and additional information as described below. Again, assuming that Bob's token indeed was "topsecret",
and that Bob is member of the "admin" and the "lamers" groups, the program should write "accept
admin lamers $uid $gid $supplementary_gids $HOME $USER\n" on its standard
output and then exit.
Thus the format of the output from an "externalvalidation" program when token validation authentication
is successful should be:
"accept $groups $uid $gid $supplementary_gids $HOME $USER\n"
Where
• $groups is a space separated list of the group names the user is a member of.
• $uid is the UNIX integer user id ConfD should use as default when executing commands for this user.
• $gid is the UNIX integer group id ConfD should use as default when executing commands for this user.
• $supplementary_gids is a (possibly empty) space separated list of additional UNIX group ids the
user is also a member of.
• $HOME is the directory which should be used as HOME for this user when ConfD executes commands
on behalf of this user.
• $USER is the user derived from mapping the token.
It is further possible for the program to return a new token on successful token validation authentication,
by using "accept_token" instead of "accept":
"accept_token $groups $uid $gid $supplementary_gids $HOME $USER $token
\n"
Where $token is an arbitrary string. ConfD will then, for some northbound interfaces, include this token
in responses.
It is also possible for the program to return additional information on successful token validation
authentication, by using "accept_info" instead of "accept":
"accept_info $groups $uid $gid $supplementary_gids $HOME $USER $info\n"
Where $info is some arbitrary text. ConfD will then just append this text to the generated audit log
message (CONFD_EXT_LOGIN).
Yet another possibility is for the program to return a warning that the user's password is about to expire,
by using "accept_warning" instead of "accept":
"accept_warning
$warning\n"

$groups

$uid

$gid

$supplementary_gids

$HOME

$USER

Where $warning is an appropriate warning message. The message will be processed by ConfD according
to the setting of /confdConfig/aaa/expirationWarning in confd.conf .

185

The AAA infrastructure

There is also support for token variations of "accept_info" and "accept_warning" namely
"accept_token_info" and "accept_token_warning". Both "accept_token_info" and
"accept_token_warning" expects the external program to output exactly the same as described
above with the addition of a token after $USER:
"accept_token_info $groups $uid $gid $supplementary_gids $HOME $USER
$token $info\n"
"accept_token_warning $groups $uid $gid $supplementary_gids $HOME $USER
$token $warning\n"
If token validation authentication failed, the program should write "reject" or "abort", possibly
followed by a reason for the rejection, and a trailing newline. For example "reject Bad password
\n" or just "abort\n". The difference between "reject" and "abort" is that with "reject",
ConfD will try subsequent mechanisms configured for /confdConfig/aaa/validationOrder
in confd.conf (if any), while with "abort", the token validation authentication fails immediately.
Thus "abort" can prevent subsequent mechanisms from being tried. Currently the only available token
validation authentication mechanism is the external one.

14.5. Group Membership
Once a user is authenticated, group membership must be established. A single user can be a member of
several groups. Group membership is used by the authorization rules to decide which operations a certain
user is allowed to perform. Thus the ConfD AAA authorization model is entirely group based. This is also
sometimes referred to as role based authorization.
All groups are stored under /nacm/groups, and each group contains a number of usernames. The
ietf-netconf-acm.yang model defines a group entry:

list group {
key name;
description
"One NACM Group Entry. This list will only contain
configured entries, not any entries learned from
any transport protocols.";
leaf name {
type group-name-type;
description
"Group name associated with this entry.";
}
leaf-list user-name {
type user-name-type;
description
"Each entry identifies the username of
a member of the group associated with
this entry.";
}
}

The tailf-acm.yang model augments this with a gid leaf:

186

The AAA infrastructure

augment /nacm:nacm/nacm:groups/nacm:group {
leaf gid {
type int32;
description
"This leaf associates a numerical group ID with the group.
When a OS command is executed on behalf of a user,
supplementary group IDs are assigned based on 'gid' values
for the groups that the use is a member of.";
}
}

A valid group entry could thus look like:


admin
bob
joe
99


The above XML data would then mean that users bob and joe are members of the admin group. The
users need not necessarily exist as actual users under /aaa/authentication/users in order to
belong to a group. If for example PAM authentication is used, it does not make sense to have all users
listed under /aaa/authentication/users.
By default, the user is assigned to groups by using any groups provided by the northbound transport (e.g. via
the confd_cli or netconf-subsys programs), by consulting data under /nacm/groups, by consulting the
/etc/group file, and by using any additional groups supplied by the authentication method. If /nacm/
enable-external-groups is set to "false", only the data under /nacm/groups is consulted.
The resulting group assignment is the union of these methods, if it is non-empty. Otherwise, the default
group is used, if configured (/confdConfig/aaa/defaultGroup in confd.conf ).
A user entry has a UNIX uid and UNIX gid assigned to it. Groups may have optional group ids. When a
user is logged in, and ConfD tries to execute commands on behalf of that user, the uid/gid for the command
execution is taken from the user entry. Furthermore, UNIX supplementary group ids are assigned according
to the gids in the groups where the user is a member.

14.6. Authorization
Once a user is authenticated and group membership is established, when the user starts to perform various
actions, each action must be authorized. Normally the authorization is done based on rules configured in
the AAA data model as described in this section, but if needed we can also register application callbacks
to partially or completely replace this logic, see the section called “AUTHORIZATION CALLBACKS”
in confd_lib_dp(3).
The authorization procedure first checks the value of /nacm/enable-nacm. This leaf has a default of
true, but if it is set to false, all access is permitted. Otherwise, the next step is to traverse the rulelist list:

list rule-list {
key "name";

187

The AAA infrastructure

ordered-by user;
description
"An ordered collection of access control rules.";
leaf name {
type string {
length "1..max";
}
description
"Arbitrary name assigned to the rule-list.";
}
leaf-list group {
type union {
type matchall-string-type;
type group-name-type;
}
description
"List of administrative groups that will be
assigned the associated access rights
defined by the 'rule' list.
The string '*' indicates that all groups apply to the
entry.";
}
// ...
}

If the group leaf-list in a rule-list entry matches any of the user's groups, the cmdrule list entries
are examined for command authorization, while the rule entries are examined for rpc, notification, and
data authorization.

14.6.1. Command authorization
The tailf-acm.yang module augments the rule-list entry in ietf-netconf-acm.yang
with a cmdrule list:

augment /nacm:nacm/nacm:rule-list {
list cmdrule {
key "name";
ordered-by user;
description
"One command access control rule. Command rules control access
to CLI commands and Web UI functions.
Rules are processed in user-defined order until a match is
found. A rule matches if 'context', 'command', and
'access-operations' match the request. If a rule
matches, the 'action' leaf determines if access is granted
or not.";
leaf name {
type string {
length "1..max";
}
description

188

The AAA infrastructure

"Arbitrary name assigned to the rule.";
}
leaf context {
type union {
type nacm:matchall-string-type;
type string;
}
default "*";
description
"This leaf matches if it has the value '*' or if its value
identifies the agent that is requesting access, i.e. 'cli'
for CLI or 'webui' for Web UI.";
}
leaf command {
type string;
default "*";
description
"Space-separated tokens representing the command. Refer
to the Tail-f AAA documentation for further details.";
}
leaf access-operations {
type union {
type nacm:matchall-string-type;
type nacm:access-operations-type;
}
default "*";
description
"Access operations associated with this rule.
This leaf matches if it has the value '*' or if the
bit corresponding to the requested operation is set.";
}
leaf action {
type nacm:action-type;
mandatory true;
description
"The access control action associated with the
rule. If a rule is determined to match a
particular request, then this object is used
to determine whether to permit or deny the
request.";
}
leaf log-if-permit {
type empty;
description
"If this leaf is present, access granted due to this rule
is logged in the developer log. Otherwise, only denied
access is logged. Mainly intended for debugging of rules.";
}
leaf comment {
type string;
description
"A textual description of the access rule.";
}

189

The AAA infrastructure

}
}

Each rule has seven leafs. The first is the name list key, the following three leafs are matching leafs. When
ConfD tries to run a command it tries to match the command towards the matching leafs and if all of
context, command, and access-operations match, the fifth field, i.e. the action, is applied.
name

name is the name of the rule. The rules are checked in order, with the ordering
given by the the YANG ordered-by user semantics, i.e. independent of
the key values.

context

context is either of the strings cli, webui, or * for a command rule. This
means that we can differentiate authorization rules for which access method is
used. Thus if command access is attempted through the CLI the context will
be the string cli whereas for operations via the Web UI, the context will be
the string webui.

command

This is the actual command getting executed. If the rule applies to one or
several CLI commands, the string is a space separated list of CLI command
tokens, for example request system reboot. If the command applies
to Web UI operations, it is a space separated string similar to a CLI string. A
string which consists of just "*" matches any command.
It is important to understand that a command rule for the CLI applies to the
string as entered by the user. The command rules are not aware of the data
model. Thus it is not possible to have a rule like:


delete-eth0
cli
delete interfaces interface eth0
exec
deny


to protect a specific interface from removal in The Juniper CLI. The user can
enter:

joe@host% edit interfaces
joe@host% delete interface eth0

making the command rule above moot.
In the Cisco like CLIs it makes more sense to use command rules to protect
data. This is due to the command oriented character of the Cisco CLIs.
In general, we do not recommend using command rules to protect the
configuration. Use rules for data access as described in the next section to
control access to different parts of the data. Command rules should be used
only for CLI commands and Web UI operations that cannot be expressed as
data rules.
Another thing that is important for command rule processing of CLI
commands is the mode. If we enable the feature /confdConfig/cli/
modeInfoInAAA, the command rule matching will match on a string where
190

The AAA infrastructure

the CLI mode is prepended. This makes command rule processing more useful
in the Cisco style CLIs than in the Juniper style CLI.
The individual tokens can be POSIX extended regular expressions. Each
regular expression is implicitly anchored, i.e. an "^" is prepended and a "$" is
appended to the regular expression.
access-operations

access-operations is used to match the operation that ConfD tries to
perform. It must be one or both of the "read" and "exec" values from the
access-operations-type bits type definition in ietf-netconfacm.yang, or "*" to match any operation.

action

If all of the previous fields match, the rule as a whole matches and the value of
action will be taken. I.e. if a match is found, a decision is made whether to
permit or deny the request in its entirety. If action is permit, the request
is permitted, if action is deny, the request is denied and an entry written
to the developer log.

log-if-permit

If this leaf is present, an entry is written to the developer log for a matching
request also when action is permit. This is very useful when debugging
command rules.

comment

An optional textual description of the rule.

For the rule processing to be written to the devel log, the /confdConfig/logs/
developerLogLevel entry in confd.conf must be set to trace.
If no matching rule is found in any of the cmdrule lists in any rule-list entry that matches the user's
groups, this augmentation from tailf-acm.yang is relevant:
augment /nacm:nacm {
leaf cmd-read-default {
type nacm:action-type;
default "permit";
description
"Controls whether command read access is granted
if no appropriate cmdrule is found for a
particular command read request.";
}
leaf cmd-exec-default {
type nacm:action-type;
default "permit";
description
"Controls whether command exec access is granted
if no appropriate cmdrule is found for a
particular command exec request.";
}
leaf log-if-default-permit {
type empty;
description
"If this leaf is present, access granted due to one of
/nacm/read-default, /nacm/write-default, or /nacm/exec-default
/nacm/cmd-read-default, or /nacm/cmd-exec-default
being set to 'permit' is logged in the developer log.
Otherwise, only denied access is logged. Mainly intended

191

The AAA infrastructure

for debugging of rules.";
}
}

• If "read" access is requested, the value of /nacm/cmd-read-default determines whether access
is permitted or denied.
• If "exec" access is requested, the value of /nacm/cmd-exec-default determines whether access
is permitted or denied.
If access is permitted due to one of these default leafs, the /nacm/log-if-default-permithas the
same effect as the log-if-permit leaf for the cmdrule lists.

14.6.2. Rpc, notification, and data authorization
The rules in the rule list are used to control access to rpc operations, notifications, and data nodes defined
in YANG models. Access to invocation of actions (tailf:action) is controlled with the same method
as access to data nodes, with a request for "exec" access. ietf-netconf-acm.yang defines a rule
entry as:
list rule {
key "name";
ordered-by user;
description
"One access control rule.
Rules are processed in user-defined order until a match is
found. A rule matches if 'module-name', 'rule-type', and
'access-operations' match the request. If a rule
matches, the 'action' leaf determines if access is granted
or not.";
leaf name {
type string {
length "1..max";
}
description
"Arbitrary name assigned to the rule.";
}
leaf module-name {
type union {
type matchall-string-type;
type string;
}
default "*";
description
"Name of the module associated with this rule.
This leaf matches if it has the value '*' or if the
object being accessed is defined in the module with the
specified module name.";
}
choice rule-type {
description
"This choice matches if all leafs present in the rule
match the request. If no leafs are present, the
choice matches all requests.";

192

The AAA infrastructure

case protocol-operation {
leaf rpc-name {
type union {
type matchall-string-type;
type string;
}
description
"This leaf matches if it has the value '*' or if
its value equals the requested protocol operation
name.";
}
}
case notification {
leaf notification-name {
type union {
type matchall-string-type;
type string;
}
description
"This leaf matches if it has the value '*' or if its
value equals the requested notification name.";
}
}
case data-node {
leaf path {
type node-instance-identifier;
mandatory true;
description
"Data Node Instance Identifier associated with the
data node controlled by this rule.
Configuration data or state data instance
identifiers start with a top-level data node. A
complete instance identifier is required for this
type of path value.
The special value '/' refers to all possible
data-store contents.";
}
}
}
leaf access-operations {
type union {
type matchall-string-type;
type access-operations-type;
}
default "*";
description
"Access operations associated with this rule.
This leaf matches if it has the value '*' or if the
bit corresponding to the requested operation is set.";
}
leaf action {
type action-type;
mandatory true;
description
"The access control action associated with the

193

The AAA infrastructure

rule. If a rule is determined to match a
particular request, then this object is used
to determine whether to permit or deny the
request.";
}
leaf comment {
type string;
description
"A textual description of the access rule.";
}
}

tailf-acm augments this with two additional leafs:
augment /nacm:nacm/nacm:rule-list/nacm:rule {
leaf context {
type union {
type nacm:matchall-string-type;
type string;
}
default "*";
description
"This leaf matches if it has the value '*' or if its value
identifies the agent that is requesting access, e.g. 'netconf'
for NETCONF, 'cli' for CLI, or 'webui' for Web UI.";
}
leaf log-if-permit {
type empty;
description
"If this leaf is present, access granted due to this rule
is logged in the developer log. Otherwise, only denied
access is logged. Mainly intended for debugging of rules.";
}
}

Similar to the command access check, whenever a user through some agent tries to access an rpc, a
notification, a data item, or an action, access is checked. For a rule to match, three or four leafs must match
and when a match is found, the corresponding action is taken.
We have the following leafs in the rule list entry.
name

name is the name of the rule. The rules are checked in order,
with the ordering given by the the YANG ordered-by user
semantics, i.e. independent of the key values.

module-name

The module-name string is the name of the YANG module where
the node being accessed is defined. The special value * (i.e. the
default) matches all modules.

Note
Since the elements of the path to a given node may be
defined in different YANG modules when augmentation
is used, rules which have a value other than * for

194

The AAA infrastructure

the module-name leaf may require that additional
processing is done before a decision to permit or deny or
the access can be taken. Thus if an XPath that completely
identifies the nodes that the rule should apply to is given
for the path leaf (see below), it may be best to leave the
module-name leaf unset.
rpc-name / notification-name / path

This is a choice between three possible leafs that are used for
matching, in addition to the module-name:
rpc-name
The name of a rpc operation, or "*" to match any rpc.
notification-name
The name of a notification, or "*" to match any notification.
path
A restricted XPath expression leading down into the populated
XML tree. A rule with a path specified matches if it is equal
to or shorter than the checked path. Several types of paths are
allowed.
1. Tagpaths that are not containing any keys. For example /
interfaces/interface/mtu .
2. Instantiated
key:
as
in
/interfaces/
interface[name="eth0"]/mask matches the mask
element only for the interface name "eth0". It's
possible to have partially instantiated paths only
containing some keys instantiated - i.e combinations
of tagpaths and keypaths. Assuming a deeper
tree, the path /hosts/host[name="venus"]/
servers/server/ip matches the "ip" element for all
servers, but only for the host named "venus".
3. Wild card at end as in: /interfaces/interface/*
does not match /interfaces/interface but rather
all children of that path.
Thus the path in a rule is matched against the path in the
attempted data access. If the attempted access has a path that
is equal to or longer than the rule path - we have a match.
If none of the leafs rpc-name, notification-name, or
path are set, the rule matches for any rpc, notification, data, or
action access.

context

context is either of the strings cli, netconf, webui, snmp,
or * for a data rule. Furthermore, when we initiate user sessions
from MAAPI, we can choose any string we want.
Similarly to command rules we can differentiate access depending
on which agent is used to gain access.

access-operations

access-operations is used to match the operation that
ConfD tries to perform. It must be one or more of the "create",

195

The AAA infrastructure

"read", "update", "delete" and "exec" values from the accessoperations-type bits type definition in ietf-netconfacm.yang, or "*" to match any operation.
action

This leaf has the same characteristics as the action leaf for
command access.

log-if-permit

This leaf has the same characteristics as the log-if-permit leaf
for command access.

comment

An optional textual description of the rule.

If no matching rule is found in any of the rule lists in any rule-list entry that matches the user's
groups, the data model node for which access is requested is examined for presence of the NACM
extensions:
• If the nacm:default-deny-all extension is specified for the data model node, access is denied.
• If the nacm:default-deny-write extension is specified for the data model node, and "create",
"update", or "delete" access is requested, access is denied.
If examination of the NACM extensions did not result in access being denied, the value (permit or
deny) of the relevant default leaf is examined:
• If "read" access is requested, the value of /nacm/read-default determines whether access is
permitted or denied.
• If "create", "update", or "delete" access is requested, the value of /nacm/write-default
determines whether access is permitted or denied.
• If "exec" access is requested, the value of /nacm/exec-default determines whether access is
permitted or denied.
If access is permitted due to one of these default leafs, this augmentation from tailf-acm.yang is
relevant:

augment /nacm:nacm {
...
leaf log-if-default-permit {
type empty;
description
"If this leaf is present, access granted due to one of
/nacm/read-default, /nacm/write-default, /nacm/exec-default
/nacm/cmd-read-default, or /nacm/cmd-exec-default
being set to 'permit' is logged in the developer log.
Otherwise, only denied access is logged. Mainly intended
for debugging of rules.";
}
}

I.e. it has the same effect as the log-if-permit leaf for the rule lists, but for the case where the value
of one of the default leafs permits the access.
When ConfD executes a command, the command rules in the authorization database are searched, The
rules are tried in order, as described above. When a rule matches the operation (command) that ConfD is
attempting, the action of the matching rule is applied - whether permit or deny.

196

The AAA infrastructure

When actual data access is attempted, the data rules are searched. E.g. when a user attempts to execute
delete aaa in the CLI, the user needs delete access to the entire tree /aaa.
Another example is if a CLI user writes show configuration aaa TAB it suffices to have read
access to at least one item below /aaa for the CLI to perform the TAB completion. If no rule matches or
an explicit deny rule is found, the CLI will not TAB complete.
Yet another example is if a user tries to execute delete aaa authentication users, we need to
perform a check on the paths /aaa and /aaa/authentication before attempting to delete the sub
tree. Say that we have a rule for path /aaa/authentication/users which is an permit rule and
we have a subsequent rule for path /aaa which is a deny rule. With this rule set the user should indeed
be allowed to delete the entire /aaa/authentication/users tree but not the /aaa tree nor the /
aaa/authentication tree.
We have two variations on how the rules are processed. The easy case is when we actually try to read or
write an item in the configuration database. The execution goes like:
foreach rule {
if (match(rule, path)) {
return rule.action;
}
}

The second case is when we execute TAB completion in the CLI. This is more complicated. The execution
goes like:
rules = select_rules_that_may_match(rules, path);
if (any_rule_is_permit(rules))
return permit;
else
return deny;

The idea being that as we traverse (through TAB) down the XML tree, as long as there is at least one rule
that can possibly match later, once we have more data, we must continue.
For example assume we have:
1. "/system/config/foo" --> permit
2. "/system/config" --> deny
If we in the CLI stand at "/system/config" and hit TAB we want the CLI to show foo as a
completion, but none of the other nodes that exist under /system/config. Whereas if we try to execute
delete /system/config the request must be rejected.

14.6.3. Authorization Examples
Assume that we have two groups, admin and oper. We want admin to be able to see and and edit the
XML tree rooted at /aaa, but we do not want users that are members of the oper group to even see
the /aaa tree. We would have the following rule-list and rule entries. Note, here we use the XML data
from tailf-aaa.yang to exemplify. The examples apply to all data, for all data models loaded into
the system.


197

The AAA infrastructure

admin
admin

tailf-aaa
tailf-aaa
/
read create update delete
permit



oper
oper

tailf-aaa
tailf-aaa
/
read create update delete
deny



If we do not want the members of oper to be able to execute the NETCONF operation edit-config,
we define the following rule-list and rule entries:

oper
oper

edit-config
edit-config
netconf
exec
deny



To spell it out, the above defines four elements to match. If ConfD tries to perform a netconf operation,
which is the operation edit-config, and the user which runs the command is member of the oper
group, and finally it is an exec (execute) operation, we have a match. If so, the action is deny.
The path leaf can be used to specify explicit paths into the XML tree using XPath syntax. For example
the following:

admin
admin

bob-password
/aaa/authentication/users/user[name='bob']/password
cli
read update
permit



198

The AAA infrastructure

Explicitly allows the admin group to change the password for precisely the bob user when the user is
using the CLI. Had path been /aaa/authentication/users/user/password the rule would
apply to all password elements for all users. Since the path leaf completely identifies the nodes that the
rule applies to, we do not need to give tailf-aaa for the module-name leaf.
ConfD applies variable substitution, whereby the username of the logged in user can be used in a path.
Thus:


admin
admin

user-password
/aaa/authentication/users/user[name='$USER']/password
cli
read update
permit



The above rule allows all users that are part of the admin group to change their own passwords only.
Finally if we wish members of the oper group to never be able to execute the request system
reboot command, also available as a reboot NETCONF rpc, we have:


oper
oper

request-system-reboot
cli
request system reboot
exec
deny




request-reboot
cli
request reboot
exec
deny


netconf-reboot
reboot
netconf
exec
deny


199

The AAA infrastructure



Debugging the AAA rules can be hard. The best way to debug rules that behave unexpectedly is to add
the log-if-permit leaf to some or all of the rules that have action permit. Whenever such a rule
triggers a permit action, an entry is written to the developer log.
Finally it is worth mentioning that when a user session is initially created it will gather the authorization
rules that are relevant for that user session and keep these rules for the life of the user session. Thus when
we update the AAA rules in e.g. the CLI the update will not apply to the current session - only to future
user sessions.

14.7. The AAA cache
ConfD's AAA subsystem will cache the AAA information in order to speed up the authorization process.
This cache must be updated whenever there is a change to the AAA information. The mechanism for this
update depends on how the AAA information is stored, as described in the following two sections.

14.8. Populating AAA using CDB
In order to start ConfD, the data models for AAA must be loaded. The defaults in the case that no actual
data is loaded for these models allow all read and exec access, while write access is denied. Access may still
be further restricted by the NACM extensions, though - e.g. the /nacm container has nacm:defaultdeny-all, meaning that not even read access is allowed if no data is loaded.
The AAA data can either be stored in CDB or in an external daemon as described in the chapter Chapter 7,
The external database API. The only new problem when we use CDB to store the AAA data is to initialize
the AAA database. This can be done as described in the chapter Chapter 5, CDB - The ConfD XML
Database, from an XML document containing real data.
ConfD ships with a decent initialization document for the AAA database. The file is called
aaa_init.xml and is by default copied to the CDB directory by the ConfD install scripts. The file
defines two users, admin and oper with passwords set to admin and oper respectively.
Normally the AAA data will be stored as configuration in CDB. This allows for changes to be made
through ConfD's transaction-based configuration management. In this case the AAA cache will be
updated automatically when changes are made to the AAA data. If changing the AAA data via ConfD's
configuration management is not possible or desirable, it is alternatively possible to use the CDB
operational data store for AAA data. In this case the AAA cache can be updated either explicitly e.g. by
using the maapi_aaa_reload() function, see the ???? manual page, or by triggering a subscription
notification by using the "subscription lock" when updating the CDB operational data store, see ????.

14.9. Populating AAA using external data
Note
The confd_aaa_bridge program described here is deprecated. It does not support the NACM
data model. It may still be useful to study this as an example of an external data provider for AAA
data, however the implementation follows the same principles as for other external data providers.
An alternative to storing the AAA data in CDB is to store it outside of ConfD. ConfD comes with an
example implementation of such a program. It is called confd_aaa_bridge and is fully described in
the man page confd_aaa_bridge(1).

200

The AAA infrastructure

The procedure here is precisely the same as with any other data model - with the exception that ConfD
itself is a user of this data and reads it. Thus if using CDB to store the AAA data is not an option, the API
for external databases must be used to populate the AAA tree.
The YANG model which describes this is the same tailf-aaa.yang but annotated with a callpoint. It
is shipped together with the example implementation called confd_aaa_bridge. When we compile the
tailf-aaa.yang with the callpoint for external data we name the resulting file aaa_bridge.fxs.

Note
The name of the fxs file is not significant, the aaa_bridge.fxs name is used here only to
distinguish it from the aaa_cdb.fxs name that has been used for the CDB version of legacy
tailf-aaa. We can equally well use the more "natural" name tailf-aaa.fxs in both cases.

Note
We must additionally annotate the ietf-netconf-acm.yang module with a callpoint and
compile it, if the group assignment and authorization data is to be provided by an external
data provider. This annotation must be done in addition to the annotation done by the ietfnetconf-acm-ann.yang module included in the ConfD release.
The example program confd_aaa_bridge.c which is delivered as source code in the ConfD release
contains an example implementation of external storage of the AAA data in an ad hoc .ini style file.
(This is only of interest for users that do not use CDB to store any data at all.) See the UNIX man page
confd_aaa_bridge(1).
The confd_aaa_bridge program implements a configuration data provider, and thus changes to
the AAA data can be made through ConfD's configuration management just as when the data is
stored as configuration in CDB. And similar to the CDB case, if changing the AAA data via ConfD's
configuration management is not possible or desirable, it is alternatively possible to let the AAA data
be provided by an operational (i.e. read-only) external data provider. In either case, when we use an
external data provider for AAA, the AAA cache must always be updated explicitly, e.g. by using the
maapi_aaa_reload() function, see the confd_lib_maapi(3) manual page. For a configuration data
provider, the confd_aaa_reload() function may be more convenient, see the confd_lib_dp(3)
manual page - this function is used by confd_aaa_bridge.

14.10. Hiding the AAA tree
Some applications may not want to expose the AAA data to end users in the CLI or the Web UI. Two
reasonable approaches exist here and both rely on the tailf:export statement. If a module has
tailf:export none it will be invisible to all agents. We can then either use a transform whereby we
define another AAA model and write a transform program which maps our AAA data to the data which
must exist in tailf-aaa.yang and ietf-netconf-acm.yang. This way we can choose to export
and and expose an entirely different AAA model.
Yet another very easy way out, is to define a set of static AAA rules whereby a set of fixed users and
fixed groups have fixed access to our configuration data. Possibly the only field we wish to manipulate
is the password field.

201

Chapter 15. The NETCONF Server
15.1. Introduction
This chapter describes the north bound NETCONF implementation in ConfD. As of this writing, the server
supports the following specifications:
• RFC 4741 - NETCONF Configuration Protocol
• RFC 4742 - Using the NETCONF Configuration Protocol over Secure Shell (SSH)
• RFC 5277 - NETCONF Event Notifications
• RFC 5717 - Partial Lock Remote Procedure Call (RPC) for NETCONF
• RFC 6020 - YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)
• RFC 6021 - Common YANG Data Types
• RFC 6022 - YANG Module for NETCONF Monitoring
• RFC 6241 - Network Configuration Protocol (NETCONF)
• RFC 6242 - Using the NETCONF Configuration Protocol over Secure Shell (SSH)
• RFC 6243 - With-defaults capability for NETCONF
• RFC 6470 - NETCONF Base Notifications
• RFC 6536 - NETCONF Access Control Model
• RFC 6991 - Common YANG Data Types
• RFC 7895 - YANG Module Library
• RFC 7950 - The YANG 1.1 Data Modeling Language

Note
For the  operation specified in RFC 4741 / RFC 6241, only  with scheme
"file" is supported for the  parameter - i.e. no data stores can be deleted. The concept
of deleting a data store is not well defined, and at odds with the transaction-based configuration
management of ConfD. To delete the entire contents of a data store, with full transactional
support, a  with an empty  element for the  parameter can be
used.

Note
For the  operation, RFC 5717, section 2.4.1 says that f a node in the scope of the
lock is deleted by the session owning the lock, it is removed from the scope of the lock. In ConfD
this is not true; the deleted node is kept in the scope of the lock.
ConfD NETCONF north bound API can be used by arbitrary NETCONF clients. A simple Python
based NETCONF client called netconf-console is shipped as source code in the distribution. See

202

The NETCONF Server

Section 15.8, “Using netconf-console” for details. Other NETCONF clients will work too, as long as they
adhere to the NETCONF protocol. If you need a Java client, the open source client JNC can be used.
When integrating NCS into larger OSS/NMS environments, the NETCONF API is a good choice of
integration point.

15.2. Capabilities
The NETCONF server in ConfD supports all capabilities in both NETCONF 1.0 (RFC 4741) and
NETCONF 1.1 (RFC 6241).
:writable-running
This capability is enabled by default. If the candidate is used, this capability should be disabled in
confd.conf(5). Additionally, /confdConfig/datastores/running/access should be set
to writable-through-candidate.
:candidate
The NETCONF server uses the candidate provided by the ConfD backplane. This can either be
implemented in an external database, or using the built-in candidate support.
This capability is enabled by default. If the candidate is not used, this capability should be disabled
in confd.conf(5).
:confirmed-commit
If the running data store is implemented as an external database, it has to support the checkpoint
functions (see Chapter 7, The external database API). If it doesn't support checkpoints, this capability
must be disabled. The built-in CDB database supports checkpoints, and can thus be used with this
capability.
This capability is enabled by default. If the candidate is not used, this capability should be disabled
in confd.conf(5).
ConfD supports both version 1.0 and 1.1 of this capability.
:rollback-on-error
This capability allows the client to set the  parameter to rollback-onerror. The other permitted values are stop-on-error (default) and continue-on-error.
Note that the meaning of the word "error" in this context is not defined in the specification. Instead,
the meaning of this word must be defined by the data model. Also note that if stop-on-error
or continue-on-error is triggered by the server, it means that some parts of the edit operation
succeeded, and some parts didn't. The error partial-operation must be returned in this case.
partial-operation is obsolete and SHOULD NOT be returned by a server. If some other error
occurs (i.e. an error not covered by the meaning of "error" above), the server generates an appropriate
error message, and the data store is unaffected by the operation.
The ConfD server never allows partial configuration changes, since it might result in inconsistent
configurations, and recovery from such a state can be very difficult for a client. This means that
regardless of the value of the  parameter, ConfD will always behave as if it had
the value rollback-on-error. So in ConfD, the meaning of the word "error" in stop-onerror and continue-on-error, is something which never can happen.
This capability is enabled by default. It can be disabled in confd.conf(5), but it doesn't affect the server
behavior, other than the capability is not advertised.
It is possible to configure the NETCONF server to generate an operation-not-supported
error if the client asks for the error-option continue-on-error. See confd.conf(5).

203

The NETCONF Server

:validate
This capability is enabled by default. It can be disabled in confd.conf(5). The only reason for disabling
this capability would be if CDB is not used, and validation constraints are not specified in the YANG
data models, and the underlying database does not support any form of validation.
ConfD supports both version 1.0 and 1.1 of this capability.
:startup
This capability is disabled by default. Enable this if /confdConfig/datastores/startup is
enabled.
:url
The URL schemes supported are file, ftp, and sftp (SSH File Transfer Protocol).
There is no standard URL syntax for the sftp scheme, but ConfD supports the syntax used by curl:
sftp://:@/

Note that user name and password must be given for sftp URLs.
This capability is disabled by default, but can be enabled in confd.conf(5).
ConfD does not support validate from a url.
:xpath
This capability is enabled by default, but can be disabled in confd.conf(5).
The NETCONF server supports XPath according to the W3C XPath 1.0 specification (http://
www.w3.org/TR/xpath), except for the list given below. There are several reasons for not supporting
conventional XPath or for diverging from XPath, including the following:
1. The operation is performed on an XML database, not an XML document.
2. The implementation context does not support the operation.
3. Immaturity of IETF specifications. This refers to the result returned for some queries.
An XPath expression evaluation may terminate without matches or with an error (returned as a
NETCONF error). Upon one or more successful matches, the XPath output is returned as an XML tree
summarizing the matched database information, similarly to a conventional NETCONF subtree filter.
The following XPath features are not available:
• Variables are not supported, since the evaluation context binds no variables.
• Some location step axes are not supported: preceding, following, preceding-sibling, followingsibling.
• Some node tests are not supported: comment(), processing-instruction(). Note that these node types
are not stored in the database.
• The XPath root node is not available. Instead, evaluation begins from each exported namespace.
This primarily affects the parent and ancestor axes.
• XPath built-ins:

204

The NETCONF Server

id()

the database does not store unique IDs

The following list of optional standard capabilities are also supported:
:notification
ConfD implements the urn:ietf:params:netconf:capability:notification:1.0
capability, including support for the optional replay feature.
This capability is disabled by default, but can be enabled in confd.conf(5).
See Section 15.7, “Notification Capability” for details.
:interleave
ConfD implements the urn:ietf:params:netconf:capability:interleave:1.0
capability, which allows the client to get send RPCs while a notification subscription is active.
This capability is disabled by default, but can be enabled in confd.conf(5).
:partial-lock
ConfD implements the urn:ietf:params:netconf:capability:partial-lock:1.0
capability, which allows the client to lock parts of the running data store.
This capability is disabled by default, but can be enabled in confd.conf(5).
:with-defaults
ConfD
implements
the
urn:ietf:params:netconf:capability:withdefaults:1.0 capability, which is used by the server to inform the client how default values are
handled by the server, and by the client to control whether defaults values should be generated to
replies or not.
This capability is enabled by default, but can be disabled in confd.conf(5).
:yang-library
ConfD implements the urn:ietf:params:netconf:capability:yang-library:1.0
capability, which informs the client that server implements the YANG module library RFC 7895,
and informs the client about the current module-set-id.
This capability is required by the YANG 1.1 specification RFC 7950, and cannot be disabled.
In addition to the standard capabilities ConfD also includes the following optional, non-standard
capabilities. They must be explicitly enabled in confd.conf (see confd.conf(5)) to be used.
actions
See Section 15.9, “Actions Capability” for details.
This capability should be enabled if actions are defined in the data model.
transactions
See Section 15.10, “Transactions Capability” for details.
This capability should be defined if ConfD runs as a subagent.
proxy forwarding
See Section 15.11, “Proxy Forwarding Capability” for details. This capability should be defined if
ConfD runs as a proxy host.

205

The NETCONF Server

inactive
See Section 15.12, “Inactive Capability” for details.
This capability should be defined if ConfD is configured to use attributes (see /confdConfig/
enableAttributes in confd.conf(5)).
The server reports each data model namespace it has loaded as separate capabilities, according to the
YANG specification.
The user can configure the server to make it report additional capability URIs.

15.2.1. Advertising Capabilities and YANG Modules
All enabled NETCONF capabilities are advertised in the hello message that the server sends to the client.
A YANG module is supported by the NETCONF server if it's fxs file is found in ConfD's loadPath, and
if the fxs files is exported to NETCONF.
The following YANG modules are built-in, which means that their fxs files must not be present in the
loadPath:
• ietf-netconf
• ietf-netconf-with-defaults
• ietf-yang-library
• ietf-yang-types
• ietf-inet-types
All built-in modules except ietf-netconf-with-defaults are always supported by the server.
Support for ietf-netconf-with-defaults can be controlled by a setting in confd.conf.
All YANG version 1 modules supported by the server are advertised in the hello message, according to
the rules defined in RFC 6020.
All YANG version 1 and version 1.1 modules supported by the server are advertised in the module list
defined in ietf-yang-library.
If a YANG module (any version) is supported by the server, and its .yang or .yin file is found in the fxs file
or in the loadPath, then the module is also advertised in the schema list defined in ietf-netconfmonitoring, made available for download with the RPC operation get-schema, and if RESTCONF
is enabled, also advertised in the schema leaf in ietf-yang-library. See Section 15.6, “Monitoring
of the NETCONF Server”.

15.3. NETCONF Transport Protocols
The NETCONF server natively supports the mandatory SSH transport, i.e., SSH is supported without the
need for an external SSH daemon (such as sshd). It also supports integration with OpenSSH.

15.3.1. Using OpenSSH
ConfD is delivered with a program netconf-subsys which is an OpenSSH subsystem program. It is invoked
by the OpenSSH daemon after successful authentication. It functions as a relay between the ssh daemon

206

The NETCONF Server

and ConfD; it reads data from the ssh daemon from standard input, and writes the data to ConfD over
a loopback socket, and vice versa. This program is delivered as source code in $CONFD_DIR/src/
confd/netconf/netconf-subsys.c . It can be modified to fit the needs of the application. For
example, it could be modified to read the group names for a user from an external LDAP server.
When using OpenSSH, the users are authenticated by OpenSSH, i.e. the user names are not stored in
ConfD. To use OpenSSH, compile the netconf-subsys program, and put the executable in e.g. /usr/
local/bin. Then add the following line to the ssh daemon's config file, sshd_config:
Subsystem

netconf

/usr/local/bin/netconf-subsys

The connection from netconf-subsys to ConfD can be arranged in one of two different ways:
1. Make sure ConfD is configured to listen to TCP traffic on localhost, port 2023, and disable SSH in
confd.conf (see confd.conf(5) ). (Re)start sshd and ConfD. Or:
2. Compile netconf-subsys to use a connection to the IPC port instead of the NETCONF TCP transport
(see the netconf-subsys.c source for details), and disable both TCP and SSH in confd.conf
. (Re)start sshd and ConfD.
This method may be preferable, since it makes it possible to use the IPC Access Check (see
Section 28.6.2, “Restricting access to the IPC port” ) to restrict the unauthenticated access to ConfD
that is needed by netconf-subsys.
By default the netconf-subsys program sends the names of the UNIX groups the authenticated user belongs
to. To test this, make sure that ConfD is configured to give access to the group(s) the user belongs to.
Easiest for test is to give access to all groups.

15.3.2. Internal TCP Transport
The server can also be configured to accept plain TCP traffic. This can be useful during development, for
debugging purposes, but it can also be used to plug in any other transport protocol. The way this works
is that some other daemon terminates the transport and authenticates the user. Then it connects to the
NETCONF server over TCP (preferably over the loopback interface for security reasons) and relays the
XML traffic to NETCONF.
In this case, the transport daemon will have to authenticate the user, and then tell the NETCONF server
about it. This should be done as a header sent over the TCP socket before any other bytes are sent. There
are two supported variants of this, only differing in encoding of the username. The first with the username
in plain text, where the header looks like this:
[username;source;proto;uid;gid;subgids;homedir;group-list;]\n

and the second with the username base64-encoded, where the header looks like this:
b64[b64username;source;proto;uid;gid;subgids;homedir;group-list;]\n

Where username is the plain text name of the authenticated user. b64username is the base64-encoded
name of the authenticated user. source is the textual representation of the ipv4 or ipv6 address and
port which the user connected from, with address and port separated by '/' (e.g. "10.0.0.1/1234"). proto
is the name of the transport protocol the client used (e.g. "beep" or "ssh"). uid, gid, supgids and
homedir are the UNIX user id, group id, supplementary group ids and home directory for this user.
These four parameters are only used if the user invokes a NETCONF RPC which is implemented with
an external program (see Section 15.5, “Extending the NETCONF Server”). group-list is a comma-

207

The NETCONF Server

separated list of group names for the user. This list should only be sent if the transport has the capability
to determine which groups a user belongs to. If not, an empty list should be sent. In this case, the normal
AAA mechanisms are used to determine group membership.
All NETCONF RPCs sent over this socket must use the framing protocol used by NETCONF over SSH.
The TCP socket is also used if we want to use a standard SSH daemon such as sshd instead of the builtin SSH implementation. Then we would configure sshd to invoke a special program for the "netconf"
subsystem. This special program would connect to the TCP socket as described above. See more below.

15.4. Configuration of the NETCONF Server
ConfD itself is configured through a configuration file called confd.conf. In that file the following
items are related to the NETCONF server. For a complete description of these parameters, please see the
confd.conf(5) man page.
/confdConfig/logs/
netconfLog

This log can be enabled in order to troubleshoot the netconf
sessions.

/confdConfig/logs/
netconfTraceLog

When this log is enabled, all NETCONF traffic to and from ConfD
is stored in a file. This can be useful in order to understand and
troubleshoot the NETCONF protocol interactions.

/confdConfig/aaa/
sshServerKeyDir

This is where the built-in SSH server reads its ssh keys.

/confdConfig/aaa/pam/
service

This is the name of the PAM service to be used by the built-in SSH
server. Used only if PAM is enabled (which means an SSH user can
log in with username and password).

/confdConfig/netconf/
enabled

When set to "true", the NETCONF server is started.

/confdConfig/netconf/
transport/ssh

Settings for the built-in SSH server, such as listen ip address and
port.

/confdConfig/netconf/
transport/tcp

Settings for the plain-text TCP transport, such as listen ip address
and port.

/confdConfig/netconf/
capabilities

Under this parameter, we can control which capabilities are
reported by the server.

/confdConfig/netconf/
capabilities/capability

This parameter can be given multiple times. It specifies a URI string
which the NETCONF server will report as a capability in the hello
message sent to the client.

15.4.1. Error Handling
When ConfD processes , , and  requests, the resulting data
set can be very large. To avoid buffering huge amounts of data, ConfD streams the reply to the client as it
traverses the data tree and calls data provider functions to retrieve the data.
If a data provider fails to return the data it is supposed to return, ConfD can take one of two actions. Either
it simply closes the NETCONF transport (default), or it can reply with an inline rpc error and continue to

208

The NETCONF Server

process the next data element. This behavior can be controlled with the /confdConfig/netconf/
rpcErrors configuration parameter (see confd.conf(5)). .
An inline error is always generated as a child element to the parent of the faulty element. For example, if
an error occurs when retrieving the leaf element "mac-address" of an "interface" the error might be:

atm1

application
operation-failed
error
Failed to talk to hardware

mac-address


...


If a get_next call fails in the processing of a list, a reply might look like this:


eth0
1500



application
operation-failed
error
Failed to talk to hardware

interface



15.5. Extending the NETCONF Server
NETCONF is an extensible protocol in the sense that new RPC operations can be defined separately from
the standard. The NETCONF server in ConfD supports this through a simple API. New operations are
typically identified with a new capability. When a new capability is implemented in this way, the name
of the capability should be added to the list of capabilities that the NETCONF server sends in its initial
 message. This list is defined in confd.conf (see confd.conf(5)).
New RPCs are defined in YANG modules.
An RPC can be implemented in three different ways:
• As an executable program which is started by ConfD for each new RPC. The XML is passed as-is from
ConfD to the program, and the resulting XML is generated by the program.
• As an executable program which is started by ConfD for each new RPC. The XML is parsed by ConfD,
and passed (in a certain format) on the command line to the program. ConfD generates an XML reply
based on the result from the program.

209

The NETCONF Server

• As a C callback function. The application registers the callback with ConfD, and ConfD invokes the
callback function when the RPC operation is received. ConfD parses the XML and passes it in a C data
structure to the callback. ConfD generates an XML reply from the return value from the callback.

15.5.1. RPC as an Executable, ConfD does not Translate
the XML
In this case, the RPC is implemented as an ordinary executable program, which communicate with ConfD
over stdin/stdout. When ConfD invokes the program, it will pass the entire XML operation on stdin. The
program is responsible for parsing the operation data and placing its reply on stdout, and then terminate
with exit status zero. ConfD wraps this reply in a  element. Note that ConfD does not
interpret the reply XML sent by the program; it merely sends the data as-is to the NETCONF client. Thus,
it is the responsibility of the program to produce a valid NETCONF XML reply. Note that a rpc reply
MUST contain one of ,  or .
A program can also be run in batch mode, which can be used to send asynchronous data to the client. In
this case, the program does not exit after having replied to the original RPC. Instead it signals that the reply
has been sent by sending a NUL byte to ConfD. ConfD will enter its main loop and listen for new requests
from the client and data from the external programs. When data is received from one source, this source is
handled, while the others are (potentially) blocked. The asynchronous data sent by the external program
must be a complete self-contained XML chunk, followed by a single NUL byte. The program can exit at
any time, the session towards the client is not terminated just because the program exits.
The maximum number of concurrently running batch processes can be set in confd.conf (see
confd.conf(5)) using the parameter /confdConfig/netconf/maxBatchProcesses. The default
is no limit.
Here's an example of an rpc operation defined in this way:

module math-rpc {
namespace "http://example.com/math/1.0";
prefix math;
import tailf-common {
prefix tailf;
}
rpc math {
tailf:exec "/usr/bin/math" {
tailf:raw-xml;
}
}
}

All these examples are available under netconf_extensions/simple_rpc in the examples
distribution.
Now suppose that the following rpc is received by the NETCONF server:

Example 15.1. Example math rpc

 $210

The NETCONF Server 23$ 


ConfD will invoke /usr/local/bin/math and pass:
 $23$ 

on stdin. The program will print the rpc-reply to stdout, and ConfD relays this data to the client.

15.5.2. RPC as an Executable, ConfD Translates the XML
In this case, the RPC is implemented as an ordinary executable program, with all XML parameters
converted by ConfD into command-line arguments to the program. If the program terminates normally
without producing any output on stdout, ConfD replies with an  rpc-reply. If the program terminates
normally and also generates data on stdout, ConfD interprets this data and passes it with  tags. If
the program terminates abnormally without producing any data, a generic operation-failed error is
returned. Finally, if the program terminates abnormally and also generates data on stdout, ConfD interprets
this data as an rpc-error, and sends the resulting XML to the client.
Here's an example of the same rpc operation as above defined this way:

module math-rpc {
namespace "http://example.com/math/1.0";
prefix math;
import tailf-common {
prefix tailf;
}
rpc math {
tailf:exec "/usr/bin/math";
input {
choice op {
container add {
leaf-list operand {
type int32;
min-elements 2;
max-elements 2;
}
}
container sub {
leaf-list operand {
type int32;
min-elements 2;
max-elements 2;
}
}
}

211

The NETCONF Server

}
output {
leaf result {
type int32;
}
}
}
}

Now suppose that the same RPC request as in Code listing 2 above is received. ConfD parses the XML
and invokes the command as:
/usr/local/bin/math add __BEGIN operand 2 operand 3 add __END

In general, the XML is flattened, and each XML element generates two strings on the command line. If
a container is received, the strings "elem-name" "__BEGIN" is generated. When the corresponding close
element is received, "elem-name" "__END" is generated. An element with a value will generate "elemname" "value". An empty element with no subelements will generate "elem-name" "__LEAF".
Next, the math program replies by printing on stdout:
result 5

The same translation rules applies to the result, and ConfD thus sends the following reply to the client:


5



15.5.3. RPC as a Callback Function
In this case, the RPC is implemented as a callback function in C, with all XML parameters converted by
ConfD into a C data structure.
Here's an example of the same rpc operation as above defined this way:

module math-rpc {
namespace "http://example.com/math/1.0";
prefix math;
import tailf-common {
prefix tailf;
}
rpc math {
tailf:actionpoint "math";
input {
choice op {
container add {
leaf-list operand {
type int32;
min-elements 2;
max-elements 2;

212

The NETCONF Server

}
}
container sub {
leaf-list operand {
type int32;
min-elements 2;
max-elements 2;
}
}
}
}
output {
leaf result {
type int32;
}
}
}
}

The code that implements this looks like this:
static void register_math(struct confd_daemon_ctx *dctx)
{
struct confd_action_cbs acb;
memset(&acb, 0, sizeof(acb));
strcpy(acb.actionpoint, "math");
acb.init = init_action; /* this function is not shown here */
acb.action = do_math;
if (confd_register_action_cbs(dctx, &acb) != CONFD_OK)
confd_fatal("Couldn't register action callbacks\n");
if (confd_register_done(dctx) != CONFD_OK)
confd_fatal("Failed to complete registration \n");
}
static int do_math(struct confd_user_info *uinfo,
struct xml_tag *name,
confd_hkeypath_t *kp,
confd_tag_value_t *params,
int nparams)
{
confd_tag_value_t reply[1];
int op1, op2, result;
/*
we know that we get exactly 4 parameters;
add | del BEGIN
operand 1
operand 2
add | del END
*/
op1 = CONFD_GET_INT32(CONFD_GET_TAG_VALUE(¶ms[1]));
op2 = CONFD_GET_INT32(CONFD_GET_TAG_VALUE(¶ms[2]));
switch (CONFD_GET_TAG_TAG(¶ms[0])) {
case math_add:
result = op1 + op2;
break;
case math_del:
result = op1 - op2;

213

The NETCONF Server

break;
}
CONFD_SET_TAG_INT32(&reply[0], math_result, result);
confd_action_reply_values(uinfo, reply, 1);
return CONFD_OK;
}

15.6. Monitoring of the NETCONF Server
RFC 6022 - YANG Module for NETCONF Monitoring defines a YANG module, ietf-netconfmonitoring, for monitoring of the NETCONF server. It contains statistics objects such as number of
RPCs received, status objects such as user sessions, and an operation to retrieve data models from the
NETCONF server.
In order to use this data model with ConfD, the fxs file (ietf-netconf-monitoring.fxs) must be
present in ConfD's loadPath. This fxs file is present in a development installation of ConfD.
This data model defines a new RPC operation, get-schema, which is used to retrieve YANG modules
from the NETCONF server. ConfD will report the YANG modules for all fxs files that are reported as
capabilities, and for which the corresponding YANG or YIN file is stored in the fxs file or found in the
loadPath. If a file is found in the loadPath, it has priority over a file stored in the fxs file. Note that by
default, the module and its submodules are stored in the fxs file by the compiler.
If the YANG (or YIN files) are copied into the loadPath, they can be stored as is or compressed with gzip.
The filename extension MUST be ".yang", ".yin", ".yang.gz", or ".yin.gz".
Also available is a Tail-f specific data model, tailf-netconf-monitoring, which augments
ietf-netconf-monitoring with additional data about files available for usage with the
 command with a file  source or target. /confdConfig/netconf/
capabilities/url/enabled and /confdConfig/netconf/capabilities/url/file/
enabled must both be set to true. If rollbacks are enabled, those files are listed as well, and they can
be loaded using .
This data model also adds data about which notification streams are present in the system, and data about
sessions that subscribe to the streams.
In order to use this data model with ConfD, the fxs file (tailf-netconf-monitoring.fxs) must
be present in ConfD's loadPath. This fxs file is present in a development installation of ConfD.
These fxs files are available in the $CONFD_DIR/etc/confd directory, and the source for them are
available in the $CONFD_DIR/src/confd/yang directory, in the distribution. The Makefile in
the latter directory can be modified as necessary, for example to compile the fxs files with a --export
parameter to confdc.

15.7. Notification Capability
This section describes how NETCONF notifications are implemented within ConfD, and how the
applications generates these events.
Central to NETCONF notifications is the concept of a stream. The stream serves two purposes. It works
like a high-level filtering mechanism for the client. For example, if the client subscribes to notifications on
the security stream, it can expect to get security related notifications only. Second, each stream may

214

The NETCONF Server

have its own log mechanism. For example by keeping all debug notifications in a debug stream, they can
be logged separately from the security stream.

15.7.1. Notification Streams
ConfD has built-in support for the well-known stream NETCONF, defined in RFC 5277. ConfD supports
the notifications defined in RFC 6470 - NETCONF Base Notifications on this stream. If the application
needs to send any additional notifications on this stream, it can do so.
It is up to the application to define which additional streams it supports. In ConfD, this is done in
confd.conf (see confd.conf(5)). Each stream must be listed, and whether it supports replay or not. An
example which defines two streams, security and debug:



security
Security related notifications
true

true
/var/log
S10M
50



debug
Debug notifications
true




The well-known stream NETCONF does not have to be listed, but if it isn't listed, it will not support replay.

15.7.2. Automatic Replay
ConfD has builtin support for logging of notifications, i.e., if replay support has been enabled for a stream,
ConfD automatically stores all notifications on disk ready to be replayed should a NETCONF client ask
for logged notifications. In the confd.conf fragment above the security stream has been setup to use the
builtin notification log/replay store. The replay store uses a set of wrapping log files on disk (of a certain
number and size) to store the security stream notifications.
The reason for using a wrap log is to improve replay performance whenever a NETCONF client asks for
notifications in a certain time range. Any problems with log files not being properly closed due to hard
power failures etc. is also kept to a minimum, i.e., automatically taken care of by ConfD.
As an alternative to the builtin notification replay store the application can roll its own. This is described
in the next sub-section.

15.7.3. Implementing Custom Replay
If a stream supports replay, the logging and replay functionality can alternatively be implemented
by the application. In order to do this, the application must register a set of callback functions

215

The NETCONF Server

with ConfD using the function confd_register_notification_stream(). The callbacks are
get_log_start_time() and replay(). The first one is called by ConfD in order to find the earliest
event time available in the log. The second one is invoked whenever a NETCONF client asks for a replay
subscription. For full details on the notification API, please see the confd_lib_dp(3) manual page.
The following example is available in full source code form in the examples directory. A single stream
interface is used, and it supports replay.
/* The notification context (filled in by ConfD) for the live feed */
static struct confd_notification_ctx *live_ctx;

struct confd_notification_stream_cbs ncb;

memset(&ncb, 0, sizeof(ncb));
ncb.fd = workersock;
ncb.get_log_times = log_times;
ncb.replay = start_replay;
strcpy(ncb.streamname, "interface");
ncb.cb_opaque = NULL;
if (confd_register_notification_stream(dctx, &ncb, &live_ctx) != CONFD_OK) {
confd_fatal("Couldn't register stream %s\n", ncb.streamname);
}
if (confd_register_done(dctx) != CONFD_OK) {
confd_fatal("Failed to complete registration\n");
}

In this simple example, we keep the replay log in memory, in an array:
struct notif {
struct confd_datetime eventTime;
confd_tag_value_t *vals;
int nvals;
};
/* Our replay buffer is kept in memory in this example.
* buffer of struct notif.
*/
#define MAX_BUFFERED_NOTIFS 4
static struct notif replay_buffer[MAX_BUFFERED_NOTIFS];
static unsigned int first_replay_idx = 0;
static unsigned int next_replay_idx = 0;

It's a circular

static struct confd_datetime replay_creation;
static int replay_has_aged_out = 0;
static struct confd_datetime replay_aged_time;

The get_log_start_time() callback simply returns the time of the first notification in the log:
static int log_times(struct confd_notification_ctx *nctx)
{
struct confd_datetime *aged;
if (replay_has_aged_out)
aged = &replay_aged_time;
else
aged = NULL;

216

The NETCONF Server

return confd_notification_reply_log_times(nctx, &replay_creation, aged);
}

When a client asks for a replay subscription, ConfD invokes the callback replay. The actual replay
notifications must not be sent from the callback. In this example, the callback allocates a replay structure,
and marks it as being active. The main loop will check for any active replays, and do the sending there.
#define MAX_REPLAYS 10
struct replay {
int active;
int started;
unsigned int idx;
struct confd_notification_ctx *ctx;
struct confd_datetime start;
struct confd_datetime stop;
int has_stop;
};
/* Keep tracks of active replays */
static struct replay replay[MAX_REPLAYS];

static int start_replay(struct confd_notification_ctx *nctx,
struct confd_datetime *start,
struct confd_datetime *stop)
{
int rnum;
for (rnum = 0; rnum < MAX_REPLAYS; rnum++) {
if (!replay[rnum].active) {
replay[rnum].active = 1;
replay[rnum].started = 0;
replay[rnum].idx = first_replay_idx;
replay[rnum].ctx = nctx;
replay[rnum].start = *start;
if (stop) {
replay[rnum].has_stop = 1;
replay[rnum].stop = *stop;
} else
replay[rnum].has_stop = 0; /* stop when caught up to live */
return CONFD_OK;
}
}
confd_notification_seterr(nctx, "Max no. of replay requests reached");
return CONFD_ERR;
}

15.7.4. Sending Notifications from an Application
Before an application can send a notification, the notification must be defined in a YANG module. In this
example, a notification link-down is defined. The notification has a single parameter if-index:
notification linkDown {
leaf ifIndex {
type leafref {
path "/interfaces/interface/ifIndex";
}
mandatory true;
}

217

The NETCONF Server

}

When the application sends an application, it uses the function confd_notification_send().
static void send_notifdown(int index)
{
confd_tag_value_t vals[3];
int i = 0;
CONFD_SET_TAG_XMLBEGIN(&vals[i], notif_linkDown,
CONFD_SET_TAG_UINT32(&vals[i],
notif_ifIndex,
CONFD_SET_TAG_XMLEND(&vals[i],
notif_linkDown,
send_notification(vals, i);

notif__ns);
index);
notif__ns);

i++;
i++;
i++;

}

static void send_notification(confd_tag_value_t *vals, int nvals)
{
int sz;
struct confd_datetime now;
struct notif *notif;
getdatetime(&now);
notif = &replay_buffer[next_replay_idx];
if (notif->vals) {
/* we're aging out this notification */
replay_has_aged_out = 1;
replay_aged_time = notif->eventTime;
first_replay_idx = (first_replay_idx + 1) % MAX_BUFFERED_NOTIFS;
free(notif->vals);
}
notif->eventTime = now;
sz = nvals * sizeof(confd_tag_value_t);
notif->vals = malloc(sz);
memcpy(notif->vals, vals, sz);
notif->nvals = nvals;
next_replay_idx = (next_replay_idx + 1) % MAX_BUFFERED_NOTIFS;
OK(confd_notification_send(live_ctx,
¬if->eventTime,
notif->vals,
notif->nvals));
}

15.8. Using netconf-console
The netconf-console program is a simple NETCONF client. It is delivered as Python source code
and can be used as-is or modified.
When ConfD has been started, we can use netconf-console to query the configuration of the
NETCONF Access Control groups:

$ netconf-console --get-config -x /nacm/groups






218

The NETCONF Server


admin
admin
private


oper
oper
public






With the -x flag an XPath expression can be specified, in order to retrieve only data matching that
expression. This is a very convenient way to extract portions of the configuration from the shell or from
shell scripts.

15.9. Actions Capability
15.9.1. Overview
This capability introduces one new rpc method which is used to invoke actions (methods) defined in the
data model. When an action is invoked, the instance on which the action is invoked is explicitly identified
by an hierarchy of configuration or state data.
Here's a simple example which resets an interface.





eth0










15.9.2. Motivation
The alternative is to use a specialized rpc method for each action. There are a couple of drawbacks with that:
• The name of the action has to be unique within the namespace. With the generic action method, the
name of the action is scoped by the element where the action is defined. For example, without a generic
action, ther might be two rpcs, 'reset-interface' and 'reset-server'. With the generic action, there are two
'reset' actions, scoped by 'interface' and 'server'.

219

The NETCONF Server

• Care must be taken to ensure that returned XML is unique within the namespace. Suppose the two
methods 'reset-interface' and 'reset-server' returns a 'status', but of different type. The element must be
called something like 'reset-interface-status' and 'reset-server-status'.
• With the generic action, it is easier to introduce intermediate NETCONF peers such as a master agent
in a master-sub agent deployment. For example, suppose there are two subagents, one which handles
interface 'eth0' and one which handles 'atm0'. When the hierarchy is excplicit in the request, the master
agent can dispatch to the correct subagent without any knowledge about the action parameters. On the
other hand, if the master agent gets a rpc 'reset-interface', it will have to parse the parameters to figure
out which subagent to send the request to.

15.9.3. Dependencies
None.

15.9.4. Capability Identifier
The actions capability is identified by the following capability string:
http://tail-f.com/ns/netconf/actions/1.0

15.9.5. New Operation: 
Description
The  operation identifies the data instance where the action is invoked, the action name, and its
parameters. If the action returns any result, it is scoped in the instance hierarchy in the reply.

Parameters
data:

A hierarchy of configuration or state data as defined by one of the device's data models. The first
part of the hierarchy defines which instance the action is invoked upon. Then comes the action
name, and any parameters it might need.
One action only can be executed within one rpc. If more than one actions are present in the rpc,
an error MUST be returned with an  set to "bad-element".

Positive Response
An action that does not return any result value, replies with the standard . If a result value is returned,
it is encapsulated in the standard  element.

Negative Response
An  element is included in the  if the request cannot be completed for any reason.

Example
Suppose we want to start a self-test on interface "eth0", and the test returns the run time (in seconds) of
the test and test status. In pseudo code
myif = find_if("eth0")
(time, status) = myif.self_test(IMMEDIATELY)

220

The NETCONF Server

Using the action RPC over NETCONF:





eth0

immediately










eth0

29
ok







15.9.6. Modifications to Existing Operations
None.

15.9.7. XML Schema
This XML Schema defines the new action rpc.










221

The NETCONF Server







15.10. Transactions Capability
15.10.1. Overview
This capability introduces four new rpc methods which are used to control a two-phase commit transaction
on the NETCONF server. The normal  operation is used to write data in the transaction, but
the modifications are not applied until an explicit  is sent.
This capability is formally defined in the YANG module "tailf-netconf-transactions".
A typical sequence of operations looks like this:
C
S
|
|
| capability exchange
|
|-------------------------->|
|<------------------------->|
|
|
|

|
|-------------------------->|
|<--------------------------|
|

|
|
|
|

|
|-------------------------->|
|<--------------------------|
|

|
|
|
| 
|
|-------------------------->|
|<--------------------------|
|

|
|
|
|

|
|-------------------------->|
|<--------------------------|
|

|
|
|

15.10.2. Dependencies
None.

15.10.3. Capability Identifier
The transactions capability is identified by the following capability string:

222

The NETCONF Server

http://tail-f.com/ns/netconf/transactions/1.0

15.10.4. New Operation: 
Description
Starts a transaction towards a configuration datastore. There can be a single ongoing transaction per session
at any time.
When a transaction has been started, the client can send any NETCONF operation, but any  or
 operation sent from the client MUST specify the same  as the ,
and any  MUST specify the same  as .
If the server receives an  or  with another , or a  with
another , an error MUST be returned with an  set to "invalid-value".
The modifications sent in the  operations are not immediately applied to the configuration
datastore. Instead they are kept in the transaction state of the server. The transaction state is only applied
when a  is received.
The client sends a  when all modifications have been sent.

Parameters
target:

Name of the configuration datastore towards which the transaction is started.

with-inactive:

If this parameter is given, the transaction will handle the "inactive" and "active"
attributes. If given, it MUST also be given in the  and 
invocations in the transaction.

Positive Response
If the device was able to satisfy the request, an  is sent that contains an  element.

Negative Response
An  element is included in the  if the request cannot be completed for any reason.
If there is an ongoing transaction for this session already, an error MUST be returned with  set to "bad-state".

Example









223

The NETCONF Server




15.10.5. New Operation: 
Description
Prepares the transaction state for commit. The server may reject the prepare request for any reason, for
example due to lack of resources or if the combined changes would result in an invalid configuration
datastore.
After a successful , the next transaction related rpc operation must be  or . Note that an  cannot be sent before the transaction is
either committed or aborted.
Care must be taken by the server to make sure that if  succeeds then the  SHOULD not fail, since this might result in an inconsistent distributed state. Thus,  should allocate any resources needed to make sure the  will succeed.

Parameters
None.

Positive Response
If the device was able to satisfy the request, an  is sent that contains an  element.

Negative Response
An  element is included in the  if the request cannot be completed for any reason.
If there is no ongoing transaction in this session, or if the ongoing transaction already has been prepared,
an error MUST be returned with  set to "bad-state".

Example







15.10.6. New Operation: 
Description
Applies the changes made in the transaction to the configuration datatore. The transaction is closed after
a .

224

The NETCONF Server

Parameters
None.

Positive Response
If the device was able to satisfy the request, an  is sent that contains an  element.

Negative Response
An  element is included in the  if the request cannot be completed for any reason.
If there is no ongoing transaction in this session, or if the ongoing transaction already has not been prepared,
an error MUST be returned with  set to "bad-state".

Example







15.10.7. New Operation: 
Description
Aborts the ongoing transaction, and all pending changes are discarded.  can be given
at any time during an ongoing transaction.

Parameters
None.

Positive Response
If the device was able to satisfy the request, an  is sent that contains an  element.

Negative Response
An  element is included in the  if the request cannot be completed for any reason.
If there is no ongoing transaction in this session, an error MUST be returned with  set to
"bad-state".

Example







15.10.8. Modifications to Existing Operations
The  operation is modified so that if it is received during an ongoing transaction, the
modifications are not immediately applied to the configuration target. Instead they are kept in the
transaction state of the server. The transaction state is only applied when a  is
received.
Note that it doesn't matter if the  is 'set' or 'test-then-set' in the , since nothing
is actually set when the  is received.

15.10.9. XML Schema
This XML Schema defines the new transaction rpcs.























226

The NETCONF Server






15.11. Proxy Forwarding Capability
15.11.1. Overview
The Proxy Forwarding capability makes it possible to forward NETCONF requests to a target host through
a proxy NETCONF server. It can be used in situations where a client does not have direct network access
to a target host:
+--------+
| Client |
+--------+
|
|
|
+--------+
| Proxy |
| server |
+--------+
/
\
/
\
/
\
+--------+
+--------+
| Proxy |
| Proxy |
| target |
| target |
+--------+
+--------+

See RFC 2663 for a definition of a proxy. This RFC defines two terms "Application Level Gateway" (ALG)
and "Proxy":
ALGs are similar to Proxies, in that, both ALGs and proxies
facilitate Application specific communication between clients
and servers. Proxies use a special protocol to communicate with
proxy clients and relay client data to servers and vice
versa. Unlike Proxies, ALGs do not use a special protocol to
communicate with application clients and do not require changes
to application clients.

A client that wants to set up a NETCONF session to a Proxy target first connects to the Proxy server,
which advertises the "forward" capability. The client issues a  RPC, with a  parameter
which specifies which Proxy target to connect to. The Proxy server sets up a NETCONF connection to
the Proxy target, and after successful authentication, replies with the Proxy target's capability list to the
client. From this point, the session is established, and any data received by the Proxy server from any side
is sent as-is (without interpretation) to the other side.
Client

Server

227

Target

The NETCONF Server

|
|
|
|
|
|
|
capability exchange
|
|
|<------------------------->|
|
|
|
|
|

|
|
|-------------------------->|
|
|<--------------------------|
|
|

|
|
|
|
|
|

|
|
|

|
|
|-------------------------->|
|
|<--------------------------|
|
|

|
|
|
|
|
|

|
|
|-------------------------->|
|
|
|
connect+authenticate
|
|
|-------------------------->|
|
|<--------------------------|
|
|

|
|

|
|
|<--------------------------|
|
|
|
|
|
data
|
|
|-------------------------->|
data
|
|
|-------------------------->|
|
|
|
|
|
data
|
|
data
|<--------------------------|
|<--------------------------|
|
|
|
|
|
|
|

Client Elements of Procedure
First, the client constructs a  rpc:
1. If the client does not know with authentication mechanism is supported by the Proxy server for the
target, or if it wants to do automatic login, it sends a  request without the "auth" parameter,
and waits for a reply.
2. If the client knows which mechanism to use, it sends a  request with the "auth/mechanism"
parameter set, and waits for a reply.
The client MAY set the "auth/initial-response" parameter.
3. Then the client waits for a reply.
4. If the reply contains the "capabilities" parameter, the proxy connection is establihed.
5. If the reply contains the "challenge" parameter, the client sends a  RPC with the
repsonse to the challenge, which it can get e.g. by prompting the user for credentials.
If the mechanism is PLAIN, the challenge is always empty.
After the  RPC is sent, the client continues from step (3).
228

The NETCONF Server

6. If the reply contains the "sasl-failure" error, with the "failure" parameter set to "invalid-mechansim",
the client continues from step (2).
7. If the reply contains the "sasl-failure" error, with the "failure" parameter set to "not-authorized", the
client continues from step (1) or aborts.
8. Otherwise, the client interprets the error and aborts.

Server Elements of Procedure
The procedure when the  RPC is received is as follows:
1. The server looks up the value of the "target" parameter in the "proxy" list in the running configuration.
If the target is not found, an "invalid-value" error is returned.
2. If the "auth" parameter is not present, and the server is configured to perform auto login, it extracts the
current user's credentials from the session, and continues from step (8).
3. If the "auth" parameter is not present, and the server is not configured to do auto login, it replies with
an error "sasl-authentication-needed", with a list of supported mechanisms.
4. If the "auth" parameter is present, the server verifies that the "mechanism" provided is supported by
the server.
Currently, the supported mechanism is "PLAIN".
If the mechanism is not supported, the server replies with a "sasl-failure" error with the "failure"
parameter set to "invalid-mechanism".
5. If the mechanism is supported, and the "initial-response" parameter is present, the server decodes the
response according to the mechanism.
If the response could not be decoded, the server replies with an "sasl-failure" error with the "failure"
parameter set to "incorrect-encoding".
If the response could be decoded, the server continues from step (8).
6. If the mechanism is supported, and the "initial-response" parameter is not present, the server replies
with a "challenge" parameter.
For PLAIN, the challenge is empty.
The server now remembers the target and mechanism, and waits to receive a 
RPC.
7. When the  RPC is received, the server decodes the "response" parameter as in (5).
If the response could be decoded, the server continues from step (8).
8. The server connects to the target with the given credentials.
If the connection fails due to communication problems, it replies with an "connection-failure" error.
If the server fails to authenticate with the given credentials, it replies with an "sasl-failure" error with
the "failure" parameter set to "not-authorized".
If the connection succeeds, the server replies with the capabilities of the target, and enters the proxying
mode.

229

The NETCONF Server

In proxying mode, the server reads data from both the client and the target, and writes any data received
to the other end, without interpreting the data. If any side of the connection is closed, the server closes
the other side.

15.11.2. Dependencies
None.

15.11.3. Capability Identifier
The proxy forwarding capability is identified by the following capability string:
http://tail-f.com/ns/netconf/forward/1.0

15.11.4. New Operation: 
Description
Starts a proxy forwarding connection to the given target, if all user credentials are given.
The server can be configured to automatically login to the target. In this case, the  rpc does not
contain any authentication parameters.

Parameters
target:

Name of the target host to connect to. The name refers to an entry in
the "proxy" list on the running configuration.

auth/mechanism:

Name of an SASL authentication mechanism to use. Currently the
"PLAIN" mechanism is supported.

auth/initial-response:

If allowed by the selected mechanism, an initial response can be given.
This saves one round-trip.
For the PLAIN mechanism, the response is a base64 encoded PLAIN
"message" as defined in section 2 of RFC 4616. The optional "authzid"
MUST NOT be present.

Positive Response
If the server was able to connect and authenticate to the target, it replies with the target's capability list,
and the server then enters proxying mode.
If the server could not fully authenticate the client, it replies with a "challenge" element. The client should
reply to the challenge with a  RPC.

Negative Response
If the server could not find the target, it replies with an "invalid-value" error.
If the client did not provide a mechanism, the server replies with a "sasl-authentication-needed" error, with
a list of available mechanisms.

230

The NETCONF Server

If the client provided an unsupported mechanism, the server replies with a "sasl-failure" error with the
"failure" parameter set to"invalid-mechanism".
If the inital response could not be decoded, the server replies with an "sasl-failure" error with the "failure"
parameter set to "incorrect-encoding".
If the server fails to connect to the target, it replies with an "connection-failure" error.
If the server fails to authenticate with the given credentials, it replies with an "sasl-failure" error with the
"failure" parameter set to "not-authorized".

Example 1.
The proxy server is configured to do automatic login:


rne-141





urn:ietf:params:netconf:base:1.0

urn:ietf:params:netconf:capability:writable-running:1.0





Example 2. Client needs to authenticate to the target:


rne-141




protocol
operation-failed
error
sasl-mechanisms


PLAIN




231

The NETCONF Server




rne-141

PLAIN
AGFkbWluAHNlY3JldA==




The decoded initial response in the auth message is:
adminsecret



urn:ietf:params:netconf:base:1.0





Example 3. Client needs to authenticate to the proxy target, but
fails:


rne-141




protocol
operation-failed
error
sasl-authentication-needed


PLAIN







rne-141

232

The NETCONF Server


PLAIN
AGFkbWluAGFlY3JldA==




The decoded initial response in the auth message is:
adminaecret

(bad passwd)



protocol
operation-failed
error
sasl-failure








15.11.5. New Operation: 
Description
Sent after receiving a challenge reply to the  request. If it succeds, the server will enter proxying
mode.

Parameters
response

For the PLAIN mechanism, the response is a base64 encoded PLAIN "message" as defined
in section 2 of RFC 4616. The optional "authzid" MUST NOT be present.

Positive Response
If the server was able to connect and authenticate to the target, it replies with the target's capability list,
and the server then enters proxying mode.
If the server could not fully authenticate the client, it replies with a "challenge" element. The client should
reply to the challenge with a  RPC.

Negative Response
If the response could not be decoded, the server replies with an "sasl-failure" error with the "failure"
parameter set to "incorrect-encoding".
If the server fails to connect to the target, it replies with an "connection-failure" error.
If the server fails to authenticate with the given credentials, it replies with an "sasl-failure" error with the
"failure" parameter set to "not-authorized".

233

The NETCONF Server

Example
Client needs to authenticate to the target:


rne-141




protocol
operation-failed
error
sasl-mechanisms


PLAIN






rne-141

PLAIN










AGFkbWluAHNlY3JldA==



The decoded response in the auth message is:
adminsecret



urn:ietf:params:netconf:base:1.0

234

The NETCONF Server






15.11.6. Modifications to Existing Operations
None.

15.11.7. Interactions with Other Capabilities
None.

15.11.8. XSD Schema
This XML Schema defines the new forwarding rpcs.






























235

The NETCONF Server












































 content when  is
"sasl-authentication-needed"
-->






236

The NETCONF Server






15.12. Inactive Capability
15.12.1. Overview
This capability is used by the NETCONF server to indicate that it supports marking nodes as being inactive.
A node that is marked as inactive exists in the data store, but is not used by the server. Any node can be
marked as inactive.
In order to not confuse clients that do not understand this attribute, the client has to instruct the server to
display and handle the inactive nodes. An inactive node is marked with an "inactive" XML attribute, and
in order to make it active, the "active" XML atribute is used.
This capability is formally defined in the YANG module "tailf-netconf-inactive".

15.12.2. Dependencies
None.

15.12.3. Capability Identifier
The inactive capability is identified by the following capability string:
http://tail-f.com/ns/netconf/inactive/1.0

15.12.4. New Operations
None.

15.12.5. Modifications to Existing Operations
A new parameter, , is added to the , , , ,
and  operations.
The  element is defined in the http://tail-f.com/ns/netconf/inactive/1.0 namespace, and
takes no value.
If this parameter is present in , , or , the NETCONF server will mark
inactive nodes with the "inactive" attribute.
If this parameter is present in  or , the NETCONF server will treat inactive
nodes as existing, so that an attempt to create a node which is inactive will fail, and an attempt to delete
a node which is inactive will succeed. Further, the NETCONF server accepts the "inactive" and "active"
attributes in the data hierarchy, in order to make nodes inactive or active, respectively.
If the parameter is present in , it MUST also be present in any , , , or  operations within the transaction. If it is not present in ,
it MUST NOT be present in any  operation within the transaction.

237

The NETCONF Server

The "inactive" and "active" attributes are defined in the http://tail-f.com/ns/netconf/inactive/1.0
namespace. The "inactive" attribute's value is the string "inactive", and the "active" attribute's value is the
string "active".

Example
This request creates an inactive interface:









Ethernet0/0
1500









This request shows the inactive interface:












Ethernet0/0
1500





This request shows that inactive data is not returned unless the client asks for it:












This request activates the interface:
This request creates an inactive interface:









Ethernet0/0









15.13. Tail-f Identification Capability
15.13.1. Overview
This capability is used by a NETCONF peer to inform the other peer about the NETCONF stack and
NETCONF client. The receiving peer can use this information in log files etc.
The information a peer may advertise is:
vendor:

The vendor of the NETCONF stack.

product:

The NETCONF product.

version:

The version of the product.

client-identity:

The identity of the user starting the session. This parameter can be the local user
name of the operator in the client tool.

239

The NETCONF Server

All these parameters are free form strings, advertised as query parameters to the capability URI, in the
 message.

15.13.2. Dependencies
None.

15.13.3. Capability Identifier
The identification capability is identified by the following capability string:
http://tail-f.com/ns/netconf/identification/1.0

15.13.4. Example
This is an example of how a client might advertise its identification information. Whitespace is added to
make the example more readable.



urn:ietf:params:netconf:base:1.1


http://tail-f.com/ns/netconf/identification/1.0?
vendor=tail-f
&product=ncs
&version=1.8
&client-identity=admin




15.13.5. ConfD
If a NETCONF client advertises this capability, ConfD picks up the information, and stores it
in the user session. The information is available to application programmers through the function
maapi_get_user_session_identification().

15.14. The Query API
The Query API consists of a number of RPC operations to start queries, fetch chunks of the result from
a query, restart a query, and stop a query.
In the installed release there are two YANG files named tailf-netconf-query.yang and tailfcommon-query.yang that defines these operations. An easy way to find the files is to run the following
command from the top directory of release installation:
$ find . -name tailf-netconf-query.yang

The API consists of the following operations:
• start-query: Start a query and return a query handle.
• fetch-query-result: Use a query handle to repeatedly fetch chunks of the result.

240

The NETCONF Server

• immediate-query: Start a query and return the entire result immediately.
• reset-query: (Re)set where the next fetched result will begin from.
• stop-query: Stop (and close) the query.
In the following examples, the following data model is used:
container x {
list host {
key number;
leaf number {
type int32;
}
leaf enabled {
type boolean;
}
leaf name {
type string;
}
leaf address {
type inet:ip-address;
}
}
}

Here is an example of a start-query operation:


/x/host[enabled = 'true']



name
100
1


An informal interpretation of this query is:
For each /x/host where enabled is true, select its name, and address, and return the result sorted
by name, in chunks of 100 results at the time.
Let us discuss the various pieces of this request.
The actual XPath query to run is specified by the foreach element. In the example below will search
for all /x/host nodes that has the enabled node set to true:

/x/host[enabled = 'true']


241

The NETCONF Server

Now we need to define what we want to have returned from the node set by using one or more select
sections. What to actually return is defined by the XPath expression.
We must also choose how the result should be represented. Basically, it can be the actual value or the path
leading to the value. This is specified per select chunk The possible result-types are: string , path ,
leaf-value and inline.
The difference between string and leaf-value is somewhat subtle. In the case of string the result
will be processed by the XPath function string() (which if the result is a node-set will concatenate all
the values). The leaf-value will return the value of the first node in the result. As long as the result is
a leaf node, string and leaf-value will return the same result. In the example above, we are using
string as shown below. At least one result-type must be specified.
The result-type inline makes it possible to return the full sub-tree of data in XML format. The data will
be enclosed with a tag: data.
Finally we can specify an optional label for a convenient way of labeling the returned data. In the
example we have the following:



The returned result can be sorted. This is expressed as XPath expressions, which in most cases are very
simple and refers to the found node set. In this example we sort the result by the content of the name node:
name

To limit the max amount of results in each chunk that fetch-query-result will return we can set
the limit element. The default is to get all results in one chunk.
100

With the offset element we can specify at which node we should start to receive the result. The default
is 1, i.e., the first node in the resulting node-set.
1

Now, if we continue by putting the operation above in a file query.xml we can send a request, using
the command netconf-console, like this:
$ netconf-console --rpc query.xml

The result would look something like this:

12345


The query handle (in this example "12345") must be used in all subsequent calls. To retrieve the result,
we can now send:


242

The NETCONF Server

12345


Which will result in something like the following:











If we try to get more data with the fetch-query-result we might get more result entries in return
until no more data exists and we get an empty query result back:



If we want to send the query and get the entire result with only one request, we can do this by using
immediate-query. This function takes similar arguments as start-query and returns the entire
result analogous fetch-query-result. Note that it is not possible to paginate or set an offset start
node for the result list; i.e. the options limit and offset are ignored.
An example request and response:


/x/host[enabled = 'true']



name
600












If we want to go back in the "stream" of received data chunks and have them repeated, we can do that with
the reset-query operation. In the example below we ask to get results from the 42:nd result entry:

12345
42


Finally, when we are done we stop the query:

12345


15.15. Meta-data in Attributes
ConfD supports three pieces of meta-data data nodes: tags, annotations, and inactive.
This feature is by default disabled, but can be enabled by setting /confdConfig/
enableAttributes to true in confd.conf (see confd.conf(5)).
An annotation is a string which acts a comment. Any data node present in the configuration can get an
annotation. An annotation does not affect the underlying configuration, but can be set by a user to comment
what the configuration does.
An annotation is encoded as an XML attribute 'annotation' on any data node. To remove an annotation,
set the 'annotation' attribute to an empty string.
Any configuration data node can have a set of tags. Tags are set by the user for data organization and
filtering purposes. A tag does not affect the underlying configuration.
All tags on a data node are encoded as a space separated string in an XML attribute 'tags'. To remove all
tags, set the 'tags' attribute to an empty string.
Annotation, tags, and inactive attributes can be present in , , , and . For example:




244

The NETCONF Server






eth0
...






15.16. Namespace for Additional Error
Information
ConfD adds an additional namespace which is used to define elements which are included in the  element. This namespace also describes which  elements the server might
generate, as part of an .




Tail-f's namespace for additional error information.
This namespace is used to define elements which are included
in the 'error-info' element.
The following are the app-tags used by the NETCONF agent:
o

not-writable
Means that an edit-config or copy-config operation was
attempted on an element which is read-only
(i.e. non-configuration data).

o

missing-element-in-choice
Like the standard error missing-element, but generated when
one of a set of elements in a choice is missing.

o

pending-changes
Means that a lock operation was attempted on the candidate
database, and the candidate database has uncommitted
changes. This is not allowed according to the protocol
specification.

o

url-open-failed
Means that the URL given was correct, but that it could not
be opened. This can e.g. be due to a missing local file, or
bad ftp credentials. An error message string is provided in

245

The NETCONF Server

the <error-message> element.
o

url-write-failed
Means that the URL given was opened, but write failed. This
could e.g. be due to lack of disk space. An error message
string is provided in the <error-message> element.

o

bad-state
Means that an rpc is received when the session is in a state
which don't accept this rpc. An example is
<prepare-transaction> before <start-transaction>






This element will be present in the 'error-info' container when
'error-app-tag' is "instance-required".







Contains an absolute XPath expression pointing to the element
which value refers to a non-existing instance.






Contains an absolute XPath expression pointing to the missing
element referred to by 'bad-element'.









This element will be present in the 'error-info' container when
'error-app-tag' is "too-few-elements" or "too-many-elements".







Contains an absolute XPath expression pointing to an
element which exists in too few or too many instances.

246

The NETCONF Server







Contains the number of existing instances of the element
referd to by 'bad-element'.







Contains the minimum number of instances that must
exist in order for the configuration to be consistent.
This element is present only if 'app-tag' is
'too-few-elems'.






Contains the maximum number of instances that can
exist in order for the configuration to be consistent.
This element is present only if 'app-tag' is
'too-many-elems'.










This attribute can be present on any configuration data node. It
acts as a comment for the node. The annotation does not affect the
underlying configuration data.






This attribute can be present on any configuration data node. It
is a space separated string of tags for the node. The tags of a
node does not affect the underlying configuration data, but can
be used by a user for data organization, and data filtering.





247

Chapter 16. The CLI agent
16.1. Overview
Confd provides three different CLI styles, one inspired by the Junos CLI (J), one inspired by the Cisco XR
CLI (C), and one inspired by the Cisco IOS CLI (I). All styles can be supported at the same time, or one
style can be chosen for a given deployment of ConfD. The default style is configured in the confd.conf
file using the style setting in the cli section.
The CLI is automatically rendered using the data model described by the yang files. This way we get an
auto-generated CLI, without any extra effort, except the design of our yang files. The auto-generated CLI
supports the following features:
• Command line history and command line editor.
• Tab completion for content of the configuration database.
• Monitoring and inspecting log files.
• Inspecting the system configuration and system state.
• Fully configuring the system.
Alias expansion is performed when a command line is entered. Aliases are part of the configuration and are
manipulated accordingly. In the J-style CLI this is done by manipulating the nodes in the alias configuration
tree. In the C- and I-style CLIs this is done by the alias command in configuration mode.
The C- and I-style CLIs automatically create modes for each list node in the yang files, and commands
for setting each parameter are generated. Actions specified in the yang files are mapped to commands in
the mode where they appear.
The J-style CLI contains commands for manipulating the configuration and actions in the yang files are
mapped into request commands in operational mode.
Even though the auto-generated CLI is fully functional it can be customized and extended in numerous
ways:
• Built-in commands can be moved, deleted and reordered.
• Confirmation prompts can be added to built-in commands.
• New commands can be implemented using the C-API and ordinary executables, and shell scripts can
be invoked from a command.
• New commands can be mounted freely in the existing command hierarchy.
• The built-in tab completion mechanism can be overridden using user defined callbacks.
• New command hierarchies can be created.
• A command timeout can be added, both a global timeout for all commands, and command specific
timeouts.
• Actions and parts of the data tree can be hidden and can later be made visible when the user enters a
password.

248

The CLI agent

In the C- and I-style CLIs some additional customizations are possible.
• The automatically generated modes can be suppressed.
• New modes can be added at internal nodes.
• The builtin show output can be replaced with an arbitrary command, either for the whole configuration
or just parts of it.
• Custom mode names can be assigned statically or dynamically through a C-callback
• Transactions can be disabled so that all CLI modifications take effect immediately. The IOS style CLI
automatically disables transactions.
How to customize and extend the auto-generated CLI is described in the clispec(5) manual page.
Tip: In the ConfD distribution there is an Emacs mode suitable for clispec editing.

16.2. The J-style CLI
16.2.1. Command Hierarchy
The CLI is built around a hierarchy of commands. This makes it possible to logically group commands.
The operational mode command hierarchy looks like this for the XR CLI:
--|||
|
|||
|
|
|
|
|
|
||||
|
|
||||
|
|
|
|
|
|
|
|-

commit
compare |- file
|- startup
configure
file --|- list
|- show
|- rename
|- delete
|- compare |
|- files
|- copy
help
id
monitor |
|- stop
|- start
ping
quit
request |- 
|- job |- stop
|- message
|- system |- logout |- user
set

249

The CLI agent

|||
|
|
|
|
|
|
|
|
|
|
|
||||-

set-path
show --||||||||
||||
source
ssh
telnet
traceroute

all
jobs
users
status
configuration
cli
cli |- history
log
notification
parser |- dump

The configure mode command hierarchy looks like this:
--||||
|
|
||
|
||||||||||||||||||
|
|||
|
|
||||||-

activate
annotate
commit |- confirmed
|- check
|- and-quit
compare |- file
|- running
deactivate
delete
edit
exit
help
hide
insert
move
load
quit
rename
revert
rollback
run
save
set
show |- parser |
|- dump
status
tag |- add
|- clear
|- del
top
unhide
up
validate
wizard adduser

250

The CLI agent

16.2.2. Two CLI modes
The ConfD CLI provides various commands for configuring and monitoring software, hardware, and
network connectivity of target devices. The CLI supports two modes: operational mode, for monitoring
the state of the device; and configure mode, for changing the state of the device.
The prompt indicates which mode the CLI is in. When moving from operational mode to configure mode
using the configure command, the prompt is changed from user@host> to user@host%. The prompts
can be configured using the prompt1 and prompt2 settings in the confd.conf file.
For example:
joe@io> configure
Entering configuration mode "private"
[ok][2006-06-02 12:31:59]
[edit]
joe@io%

16.2.3. Operational mode
Operational mode is the initial mode after successful login to the CLI. It is primarily used for viewing
the system status, controlling the CLI environment, monitoring and troubleshooting network connectivity,
and initiating the configure mode.
The full list of commands available in operational mode is listed below in the "Operational mode
commands" section.

16.2.4. Configure mode
Configure mode can be initiated by entering the configure command in operational mode. All changes to
the device's configuration are done to a copy of the active configuration, called a candidate configuration.
These changes do not take effect until a successful commit or commit confirm command is entered.
The full list of commands available in configure mode is listed below in the "Configure mode commands"
section.

16.3. The C- and I-style CLI
The C- and I-style CLI is inspired by the Cisco XR CLI and Cisco IOS CLI. The configuration is
manipulated through a series of commands and modes. Each parameter in the yang files is represented
by a separate command.
The CLI provides various commands for configuring and monitoring software, hardware, and network
connectivity of target devices. The CLI supports two modes: operational mode, for monitoring the state
of the device; and configure mode, for changing the state of the device. The configure mode consists
of a number of sub-modes for manipulating different parts of the configuration, i.e. the mode aaa
authentication users user is present for configuring user authentication parameters.
The prompt indicates which mode the CLI is in. When moving from operational/EXEC mode to configure
mode using the config command, the prompt is changed from host# to host(mode)#. The prompts can
be configured using the cPrompt1 and cPrompt2 settings in the confd.conf file, and additionally
in the IOS style through the prompt setting in the AAA configuration.
For example:

251

The CLI agent

joe connected from 127.0.0.1 using console on io
io# config terminal
Entering configuration mode private
io(config)#

16.3.1. Operational/EXEC mode
Operational mode is the initial mode after successful login to the CLI. It is primarily used for viewing
the system status, controlling the CLI environment, monitoring and troubleshooting network connectivity,
and initiating the configure mode.
The full list of commands available in operational mode is listed below in the "Operational mode
commands" section.

16.3.2. Configure mode
Configure mode can be initiated by entering the config command in operational mode. All changes to the
device's configuration are done to a copy of the active configuration, called a candidate configuration.
These changes do not take effect until a successful commit or commit confirm command is entered.
The full list of commands available in configure mode is listed below in the "Configure mode commands"
section. Additional commands and modes are dynamically derived from the yang files.

16.4. The CLI in action
16.4.1. Starting the CLI
The CLI is started using the confd_cli program. It can either be used as a login program or started manually
once the user has logged in.
In a typical device, ordinary users would have the confd_cli program as login shell, and the root user
would have to login and then start the CLI using confd_cli.
confd_cli is a fairly small program (about 800 lines of C code) which we distribute both as a binary
and source code (in $CONFD_DIR/src/confd/cli). When started it sets the terminal in raw mode,
connects to the ConfD daemon through a socket over the loopback interface, and sends user name, the
user's groups, protocol, client IP address, terminal settings etc., and then enters a proxying mode where it
sends key strokes from the user and prints characters sent by the ConfD daemon.
It is straightforward to modify the C program to send, for example, custom group information. The default
behavior is to send the UNIX groups the user belongs to.
Out of the box, the confd_cli program supports a range of options, primarily intended for debug and
development purposes.
The confd_cli program can also be used for batch processing of CLI commands, either by storing the
commands in a file and running confd_cli on the file, or by having the following line at the top of the file
(with the location of the program modified appropriately):
#!/bin/confd_cli

When the CLI is run non-interactively it will terminate at the first error and will only show the output of
the commands executed. It will not output the prompt or echo the commands. This is the same behavior
as for shell scripts.

252

The CLI agent

If you want to run a script non-interactively, e.g. as a script or through a pipe, and still want the prompts
and commands echoed, you can give the --interactive option.
Command line options:
confd_cli --help
Usage: confd_cli [options] [file]
Options:
--help, -h
display this help
--host, -H 
current host name (used in prompt)
--address, -A  cli address to connect to
--port, -P 
cli port to connect to
--cwd, -c 
current working directory
--proto, -p 
type of connection (tcp, ssh, console)
--verbose, -v
verbose output
--ip, -i
clients source ip
--interactive, -n
force interactive mode
--juniper, -J
Juniper style CLI
--cisco, -C
Cisco XR style CLI
--user, -u 
clients user name
--uid, -U 
clients user id
--groups, -g  clients group list
--gids, -D 
clients group id list
--gid, -G 
clients group id
--noaaa
disable AAA

host

The argument to host should be the host name of the device. The confd_cli program
will use the result of the system call gethostname() as default value. The host name
is used in the CLI prompt.

address

If ConfD has been configured to listen to a different address than 127.0.0.1 for the
communication between subsystems, then that address should be given as argument
to address (or we can use the CONFD_IPC_ADDRESS environment variable, or
recompile the confd_cli program with the new address compiled in).

port

If ConfD has been configured to use a non-default port for the communication between
subsystems, then that port number should be given as argument to port (or we can
use the CONFD_IPC_PORT environment variable, or recompile the confd_cli program
with the new port compiled in).

cwd

Directory to use as current working directory in the CLI. Normally the user's home
directory. The default is the directory where the confd_cli program is started.

proto

Should be the protocol used by the user to connect to the box, one of tcp, ssh, and
console. The default is ssh for connections established with OpenSSH (the program
inspects the SSH_CONNECTION environment variable), and console for everything
else. This value is printed in the audit logs.

verbose

If this argument is given, then the confd_cli program will be a bit more talkative during
the ConfD handshake phase.

ip

Should be the user's source IP address, if the user connects through SSH or telnet. The
default is 127.0.0.1 to indicate the console. This value is printed in the audit logs.

interactive

Force the CLI to echo commands and prompts even when not invoked from a terminal,
i.e. when reading input from a file or through a pipe.

cisco

Force the CLI to provide a I-style CLI.

253

The CLI agent

juniper

Force the CLI to provide a Juniper style CLI.

user

The name of the user connecting. Used to set proper access rules and assign proper
groups (if the group mapping is kept in ConfD). The default is to use the login name
of the user.

uid

The numeric user id of the connected user. The uid will be used when executing
osCommands, when checking file access permissions, and when creating files. Note
that ConfD needs to run as root for this to work properly.

gid

The numeric group id of the connected user. The gid will be used when executing
osCommands, when checking file access permissions, and when creating files. Note
that ConfD needs to run as root for this to work properly.

groups

The argument to groups should be a comma-separated list of groups. The default is to
send the OS groups that the user belongs to, i.e. the same as the groups shell command
gives us.

gids

The argument to gids should be a comma-separated list of numeric group ids
representing the Unix supplementary groups for the user. These are used when
executing osCommands and when checking file access permissions.

noaaa

Disables AAA. This is useful during development but should be removed in a
production system.

16.4.2. Starting the CLI in an overloaded system
If the number of ongoing sessions have reached the configured system limit, no more CLI sessions will
be allowed until one of the existing sessions have been terminated.
This makes it impossible to get into the system. A situation which may not be acceptable. The CLI therefore
has a mechanism for handling this problem. When the CLI detects that the session limit has been reached
it will check if the new user has privileges to execute the logout command. If the user does it will display
a list of the current user sessions on the box and ask the user if one of the sessions should be terminated
to make room for the new session.

16.5. Environment for OS command execution
All OS commands, i.e. executables or shell scripts, are executed using a program called cmdptywrapper.
To be able to execute an os-command as root or a specific user we need to make cmdptywrapper setuid
root, i.e.
# chown root cmdptywrapper
# chmod u+s cmdptywrapper

Failing that, all programs will be executed as the user who started the ConfD daemon. Consequently, if
that user is root we do not have to perform the chmod operation above.

16.6. Command output processing
It is possible to process the output from a command using an output redirect. This is done using the |
character. This redirect feature is supported in both the C- and I-style and the J-style CLI. Most of the
commands are the same but some commands differ. The redirect targets (pipe commands) can be modified

254

The CLI agent

using the clispec file, just as a regular CLI command. The commands can be chained to achieve more
complex processing.
In the J-style CLI the commands are called - append, count, display set (show configuration only), display
annotations, display json, display keypath, display xml, display xpath, show tags, hide annotations, hide
tags, except (exclude), extended, find (begin), linnum, match (include), match-all, match-any, more,
nomore, notab (auto-rendered show commands only), repeat (auto-rendered show commands only), save,
tab (show commands only) and until. It may look look this in the J-style CLI:

admin@tellus> show configuration | ?
Possible completions:
annotation - Show only statements whose annotation matches a pattern
append
- Append output text to a file
best-effort - Display data even if data provider is unavailable or continue loading from f
count
- Count the number of lines in the output
details
- Display details
display
- Display options
except
- Show only text that does not matches a pattern
extended
- Show referring elements
find
- Search for the first occurrence of a pattern
hide
- Hide display options
linnum
- Enumerate lines in the output
match
- Show only text that matches a pattern
match-all
- All selected filters must match
match-any
- At least one filter must match
more
- Paginate output
nomore
- Suppress pagination
save
- Save output text to a file
select
- Select additional columns
sort-by
- Select sorting indices
tab
- Enforce table output
tags
- Show only statements whose tags matches a pattern
until
- Display until the first occurrence of a pattern

In the C- and I-style CLIs the commands are called - append, count, exclude (except), display annotations,
display tags, hide annotations, hide tags, begin (find), include (match), linnum, match-all, match-any,
more, nomore, notab (auto-rendered show commands only), repeat (auto-rendered show commands only),
save, tab (show commands only) and until. It may look look this in the C- and I-style CLI:
tellus# show running-config | ?
Possible completions:
annotation
Show only statements whose annotation matches a pattern
append
Append output text to a file
begin
Begin with the line that matches
count
Count the number of lines in the output
details
Display commit progress
display
Display options
exclude
Exclude lines that match
extended
Display referring entries
hide
Hide display options
include
Include lines that match
linnum
Enumerate lines in the output
match-all
All selected filters must match
match-any
At least one filter must match
more
Paginate output
nomore
Suppress pagination
save
Save output text to a file
select
Select additional columns
sort-by
Select sorting indices

255

The CLI agent

tab
tags
until

Enforce table output
Show only statements whose tags matches a pattern
End with the line that matches

The show annotations/tags and hide annotations/tags pipe targets are only available when viewing the
configuration, and only if attributes have been enabled in the confd.conf file.

16.6.1. Sort the the Output
The sort-by target makes it possible for the CLI user to control in which order instances should be
displayed, and can be used when the path points to a list. The argument to sort-by can either be a secondary
index or an arbitrary set of leafs in the list. If a secondary index is given as an argument, the table will be
sorted in the order defined by the secondary index. If a set of leafs is given as an argument, the table will
be sorted in the order in which the leafs are entered. For example:
admin@io 13:28:07> show configuration server | sort-by port ip | tab
NAME IP
PORT DESCRIPTION
----------------------------------1
1.1.1.1
1010 7
1.1.1.17 1020 10
1.1.1.11 1040 3
1.1.1.3
1070 6
1.1.1.4
1070 5
1.1.1.5
1070 4
1.1.1.7
1070 8
1.1.1.8
1070 9
1.1.1.9
1070 11
1.1.1.10 1070 2
1.1.1.12 1070 [ok][2013-08-31 13:49:44]
admin@io 13:50:12>

16.6.2. Count the Number of Lines in the Output
This redirect target counts the number of lines in the output. For example:
admin@io 13:28:07> show configuration | count
[ok][2007-08-31 13:49:44]
Count: 99 lines
admin@io 13:49:44> show configuration aaa | count
[ok][2007-08-31 13:50:12]
Count: 90 lines
admin@io 13:50:12>

16.6.3. Search for a String in the Output
The match target (include in C- and I- style) is used to only include lines matching a regular expression.
For example:
admin@io 13:53:59> show configuration aaa | match {
aaa {
authentication {
users {
user admin {

256

The CLI agent

user oper {
user private {
user public {
groups {
group admin {
group oper {
authorization {
cmdrules {
cmdrule 1 {
cmdrule 2 {
cmdrule 3 {
cmdrule 150 {
datarules {
datarule 101 {
datarule 203 {

In the example above only lines containing { are shown. Similarly lines not containing a regular expression
can be included. This is done using the except target (exclude in C- and I- style). For example:
admin@io 13:56:30> show configuration aaa authentication | except {
uid
1000;
gid
100;
password
$1$fB$0w68PmacQ4VmE3/M3nK3Ug==;
ssh_keydir /var/confd/homes/admin/.ssh;
homedir
/var/confd/homes/admin;
}
uid
1000;
gid
100;
password
$1$S6$brGZW9wSDifHoU7Rf5KSHA==;
ssh_keydir /var/confd/homes/oper/.ssh;
homedir
/var/confd/homes/oper;
}
uid
1000;
gid
100;
password
$1$L4$YcCoIivO4mrzoj8vCrEjlw==;
ssh_keydir /var/confd/homes/private/.ssh;
homedir
/var/confd/homes/private;
}
uid
1000;
gid
100;
password
$1$Ft$9zTEc79NWFE0E8v7I2RxVQ==;
ssh_keydir /var/confd/homes/public/.ssh;
homedir
/var/confd/homes/public;
}
}
users "admin private";
}
users "oper public";
}
}
}

It is also possible to display the output starting at the first match of a regular expression, using the find
target (begin in C- and I- style). For example:
admin@io 14:03:44> show configuration aaa authentication users | find private
user private {
uid
1019;
gid
1013;
password
$1$AO$hbQEgdGQLzlWhX/1FNL5f.;

257

The CLI agent

ssh_keydir /var/confd/homes/private/.ssh;
homedir
/var/confd/homes/private;
}
user public {
uid
gid
password
ssh_keydir
homedir
}

1019;
1013;
$1$Kh$0Lor2g1yrSQ7MYDLxFr9h0;
/var/confd/homes/public/.ssh;
/var/confd/homes/public;

Output can also be ended when a line matches a regular expression. This is done with the until target.
For example:

admin@io 14:03:44> show configuration aaa authentication users | find private | until public
user private {
uid
1019;
gid
1013;
password
$1$AO$hbQEgdGQLzlWhX/1FNL5f.;
ssh_keydir /var/confd/homes/private/.ssh;
homedir
/var/confd/homes/private;
}
user public {

It is also possible to filter the output by using a sequence of select statements followed by match-any or
match-any. Consider the configuration:
admin@io
server a
ip
port
}
server b
ip
port
}
server c
ip
port
}

14:03:44> show configuration servers server
{
1.2.3.4;
23;
{
2.3.4.5;
24;
{
3.4.5.6;
25;

If we were to show all servers that has either ip 1.2.3.4 or port 24, this can be done by using select
statements, like so
admin@io
server a
ip
port
}
server b
ip
port
}

14:03:44> show configuration servers server | select ip 1.2.3.4 | select port 24 |
{
1.2.3.4;
23;
{
2.3.4.5;
24;

whereas a match-all filtering would in this case result in
admin@io 14:03:44> show configuration servers server | select ip 1.2.3.4 | select port 24 |
No entries found.

as there are no servers that has both ip 1.2.3.4 and port 24.

258

The CLI agent

16.6.4. Saving the Output to a File
The output can also be saved to a file using the save or append redirect target. For example:
admin@io 14:03:51> show configuration aaa

| save /tmp/saved

Or to save the configuration, except all passwords
admin@io 14:03:51> show configuration | except password | save /tmp/saved

16.6.5. Regular expressions
The regular expressions is a subset of the regular expressions found in egrep and in the AWK programming
language. Some common operators are:
.

Matches any character.

^

Matches the beginning of a string.

$

Matches the end of a string.

[abc...]

Character class, which matches any of the characters abc... Char- acter ranges are specified
by a pair of characters separated by a -.

[^abc...]

negated character class, which matches any character except abc....

r1 | r2

Alternation. It matches either r1 or r2.

r1r2

Concatenation. It matches r1 and then r2.

r+

Matches one or more rs.

r*

Matches zero or more rs.

r?

Matches zero or one rs.

(r)

Grouping. It matches r.

For example, to only display uid and gid you can do the following:
admin@io 15:11:24> show configuration | match "(uid)|(gid)"
uid
1000;
gid
100;
uid
1000;
gid
100;
uid
1000;
gid
100;
uid
1000;
gid
100;

16.6.6. Display line numbers
The linnum target causes a line number to be displayed at the beginning of each line in the display.
admin@io 15:11:24> show configuration | match "(uid)|(gid)" | linnum
1:
uid
1019;

259

The CLI agent

2:
3:
4:
5:
6:
7:
8:

gid
uid
gid
uid
gid
uid
gid

1013;
1019;
1013;
1019;
1013;
1019;
1013;

16.6.7. Display as format
The display target can be used to display output in a set of output formats. Some of these output formats are
unique to specific modes, such as configuration or operational mode. The output formats json, keypath,
xml, and xpath are available in most modes and CLI styles (J, I, and C).
For instance, assuming we have a data model featuring a set of hosts, each containing a set of servers, we
can display the configuration data as JSON. This is depicted in the example below.
admin@io 15:11:24> show configuration hosts | display json
{
"data": {
"pipetargets_model:hosts": {
"host": [
{
"name": "host1",
"enabled": true,
"numberOfServers": 2,
"servers": {
"server": [
{
"name": "serv1",
"ip": "192.168.0.1",
"port": 5001
},
{
"name": "serv2",
"ip": "192.168.0.1",
"port": 5000
}
]
}
},
{
"name": "host2",
"enabled": false,
"numberOfServers": 0
...

Still working with the same data model as used in the example above, we might want to see the current
configuration in keypath format. The following example shows how to do that, and shows the resulting
output.
admin@io 15:11:24> show configuration hosts | display keypath
/hosts/host{host1} enabled
/hosts/host{host1}/numberOfServers 2
/hosts/host{host1}/servers/server{serv1}/ip 192.168.0.1
/hosts/host{host1}/servers/server{serv1}/port 5001
/hosts/host{host1}/servers/server{serv2}/ip 192.168.0.1
/hosts/host{host1}/servers/server{serv2}/port 5000
/hosts/host{host2} disabled

260

The CLI agent

/hosts/host{host2}/numberOfServers 0

16.7. Range expressions
It is possible to modify a range of instances at the same time using range expressions, and to display a
range of instances using a range expression.
Key attributes that are integers are automatically support range expressions, both in the J- style and the Iand C- style CLI. The syntax is slightly different in the J-style than in I- and C- style.
Automatic range expressions are also supported for key elements of other types as long as they are restricted
to the pattern [a-zA-Z-]*[0-9]+/[0-9]+/[0-9]+/.../[0-9]+. I.e., the CLI understands the [integer]/[integer]
syntax.
If you have more complicated data-types then you may need to write a custom range callback. See the
examples.confd/cli/custom_ranges example.
Suppose we have data model like this:
module range {
namespace "http://tail-f.com/ns/example/range";
prefix range;
import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
typedef interface-type {
type string {
pattern "((FastEthernet-)|(GigaEthernet-))[0-9]+/[0-9]+/[0-9]+";
}
}
list server {
key name;
max-elements 64;
leaf name {
type int32;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf port {
type inet:port-number;
mandatory true;
}
leaf description {
type string;
}
}
list interface {
key name;

261

The CLI agent

max-elements 64;
leaf name {
type interface-type;
}
leaf ip {
type inet:ip-address;
mandatory true;
}
leaf mtu {
type int32;
mandatory true;
}
leaf description {
type string;
}
tailf:action reset {
tailf:actionpoint reset-point;
input {
leaf mode {
type string;
mandatory true;
}
leaf debug {
type empty;
}
}
output {
leaf time {
type string;
mandatory true;
}
}
}
}
}

Then you can do the following in the J-style CLI. Display a selected subset of the servers:
show configuration server 1-3,5

Display the configuration of a subset of the interfaces:
show configuration interface FastEthernet-1/1/1,2

In configure mode you can edit a range of interfaces in one go. For example:
jb@io> configure
Entering configuration mode private
[ok][2009-06-16 12:57:59]
[edit]
jb@io% edit interface FastEthernet-1/1/1-3
[ok][2009-06-16 12:58:14]
[edit interface FastEthernet-1/1/1-3]
jb@io% set mtu 1400
[ok][2009-06-16 12:58:20]
[edit interface FastEthernet-1/1/1-3]

262

The CLI agent

jb@io% commit
Commit complete.
[ok][2009-06-16 12:58:22]
[edit interface FastEthernet-1/1/1-3]
jb@io%

In the C- and I- style CLIs the syntax is slightly different in configure mode. When you want to modify
a range of instances you enter a specific range mode.
io# config
Entering configuration mode terminal
io(config)# interface range FastEthernet-1/1/1-3
io(config-interface-FastEthernet-1/1/1-3)# mtu 1300
io(config-interface-FastEthernet-1/1/1-3)# commit
Commit complete.
io(config-interface-FastEthernet-1/1/1-3)#

See the examples.confd/cli/ranges example.

16.8. Autorendering of enabled/disabled
If the data model contains an element called enabled of type xs:boolean then it will get some special
treatment. Normally the user would have to enter enabled true and enabled false to enable/disable. To
make this a bit more user friendly the CLI will also accept enabled and disabled. A disabled command
will be auto generated if there is an enabled option of the proper type.
However, the disabled command will not be visible in configuration dumps, instead enabled false will
be shown. The disabled command is only provided as a sugar in the CLI.
This behaviour can be controlled on a global level by configuring the confd.conf setting /confdConfig/
cli/useShortEnabled. If the value is set to 'true', it is possible to disable the behavior on a per leaf basis,
by using the extension tailf:cli-suppress-shortenabled.

16.9. Actions
Actions are invoked differently in C/I- and J-mode. In C/I-mode an action appears as a mode specific
command, whereas in J-mode an action is invoked using the request command in operational mode.
For example, given the data model fragment below:
tailf:action shutdown {
tailf:actionpoint actions;
input {
tailf:constant-leaf flags {
type uint64 {
range "1 .. max";
}
tailf:constant-value 42;
}
leaf timeout {
type xs:duration;
default PT60S;
}
leaf message {
type string;
}

263

The CLI agent

container options {
leaf rebootAfterShutdown {
type boolean;
default false;
}
leaf forceFsckAfterReboot {
type boolean;
default false;
}
leaf powerOffAfterShutdown {
type boolean;
default true;
}
}
}
}

In J-mode the restart action is invoked as follows:
joe@io> request shutdown timeout 10s message reboot options { forceFsckAfterReboot true }

And in C/I-mode as a mode specific command as follows:
io(config)# system restart mode quick restore { db startup

fast } debug timeout 45

Full command and argument completion is available when entering the command.
The order in which the action arguments are entered is not important. The parameters are reordered by the
CLI-backend before invoking the action callback.
In the CLI the action is not paginated by default and will only do so if it is piped to more.

joe@io> example_action | more

16.10. Command history
By default, command history is maintained in each mode. When we enter configure mode for the first
time, we will get an empty history, i.e. we cannot access the command history from operational mode.
When we exit back into operational mode we will again have access to the command history from the
preceding operational mode session. There is an exception to this, specifically, the 'confd.conf' setting
'unifiedHistory'. This makes the system use one single history for both modes.

16.11. Clearing history
By default, clearing the history in one mode (for instance operational mode) will not the affect the history
of the other mode. Enabling the 'confd.conf' setting 'unifiedHistory', will cause the 'clear history' command
to clear both the operational mode session's history and the configuration mode's history.

16.12. Command line editing
The default key strokes for editing the command line and moving around the command history are as
follows. Note that it is possible to change these commands using the keymap clispec modification. See
the clispec.5 man page for more details.

264

The CLI agent

16.12.1. Moving the cursor:
Move the cursor back one character
Ctrl-b or Left Arrow
Move the cursor back one word
Esc-b or Alt-b
Move the cursor forward one character
Ctrl-f or Right Arrow
Move the cursor forward one word
Esc-f or Alt-f
Move the cursor to the beginning of the command line
Ctrl-a or Home
Move the cursor to the end of the command line
Ctrl-e or End

16.12.2. Delete characters:
Delete the character before the cursor
Ctrl-h, Delete, or Backspace
Delete the character following the cursor
Ctrl-d
Delete all characters from the cursor to the end of the line
Ctrl-k
Delete the whole line
Ctrl-u or Ctrl-x
Delete the word before the cursor
Ctrl-w, Esc-Backspace, or Alt-Backspace
Delete the word after the cursor
Esc-d or Alt-d

16.12.3. Insert recently deleted text:
Insert the most recently deleted text at the cursor
Ctrl-y

16.12.4. Display previous command lines:
Scroll backward through the command history
Ctrl-p or Up Arrow
Scroll forward through the command history
Ctrl-n or Down Arrow
Search the command history in reverse order
Ctrl-r

265

The CLI agent

Show a list of previous commands
run the "show cli history" command

16.12.5. Capitalization:
Capitalize the word at the cursor, i.e. make the first character uppercase and the rest of the word lowercase.
Esc-c
Change the word at the cursor to lowercase.
Esc-l
Change the word at the cursor to uppercase.
Esc-u

16.12.6. Special:
Abort a command/Clear line
Ctrl-c
Quote insert character, i.e. do not treat the next keystroke as an edit command.
Ctrl-v/ESC-q
Redraw the screen
Ctrl-l
Transpose characters
Ctrl-t
Enter multi-line mode. This lets you enter multi-line values when prompted for a value in the CLI. It is
not available when editing a CLI command.
ESC-m
Exit configuration mode. Only in C- and I-style.
Ctrl-z

16.13. Using CLI completion
We do not always have to type the full command or option name for the CLI to recognize it. To display
possible completions, type the partial command followed immediately by  or .
If the partially typed command uniquely identifies a command, the full command name will appear.
Otherwise a list of possible completions is displayed.
Completion is disabled inside quotes; in other words, if we want to type an argument containing spaces,
we can either quote them with a \ (e.g. show file foo\ bar) or with a " (e.g. show file "foo bar"). Space
completion is disabled when entering a filename.
Command completion also applies to filenames and directories.
Example:
admin@io> 
Possible completions:
commit
- Confirm a pending commit
configure - Manipulate software configuration information
file
- Perform file operations

266

The CLI agent

help
- Provide help information
monitor
- Real-time debugging
ping
- Ping a host
quit
- Exit the management session
request
- Make system-level requests
set
- Set CLI properties
set-path
- Set relative show-path
show
- Show information about the system
ssh
- Open a secure shell on another host
telnet
- Open a telnet session to another host
traceroute - Trace the route to a remote host
admin@io> request 
Possible completions:
job
- Job operations
message - Send message to terminal of one or all users
system - System operations
admin@io> request message all "hello"

16.13.1. Customizing CLI completion
CLI completion kicks in for both command parameters and for values adhering to leaf elements in yang. By
default completions are generated automatically depending on the parameter or leaf element type. Some
examples:
d199# history ?


Above we trigger a ?-completion for the built-in history command which takes a single parameter of type
xs:nonNegativeInteger. The built-in history command chooses to answer with a  completion string.
d199# config
Entering configuration mode terminal
d199(config)# interface ?
Possible completions:


Above we trigger a ?-completion for the list element interface which has a key leaf element of type
interfaceNameType defined in the http://tail-f.com/ns/example/config/1.0 namespace:
typedef interfaceNameType {
tailf:info "GigaEthernetX/Y";
type string;
}

If the default completion behavior is not satisfactory a custom completion callback can be used instead.
This holds true for both built-in and user defined commands as well for leaf elements in built-in and user
defined data models.
In the following examples we redefined the completion behavior for the two examples above.
d199# history ?
Possible completions:
500 750 The history must be a non-negative value (preferably 500 or 750)

and
d199# config
Entering configuration mode terminal
d199(config)# interface ?

267

The CLI agent

Possible completions:
FastEthernet0/1
FastEthernet IEEE 802.3
GigaEthernet0/1
GigabitEtherenet IEEE 802.3z
GigaEthernet0/2
GigabitEtherenet IEEE 802.3z
GigaEthernet1/1
GigabitEtherenet IEEE 802.3z
GigaEthernet1/2
GigabitEtherenet IEEE 802.3z
d199(config)# interface GigaEthernet ?
Possible completions:
GigaEthernet0/1 GigaEthernet0/2 GigaEthernet1/1

GigaEthernet1/2

This was achieved using the following clispec fragment:




generic-complete






ifs-complete




The generic-complete and ifs-complete callbacks look like this (fragments):
static int generic_complete(struct confd_user_info *uinfo, int cli_style,
char *token, int completion_char,
confd_hkeypath_t *kp, char *cmdpath,
char *cmdparam_id,
struct confd_qname *simpleType, char *extra) {
char keypath[BUFSIZ] = {0};
struct confd_completion_value values[6];
int i = 0;

if (strcmp(cmdpath, "history") == 0 ||
strcmp(cmdpath, "set history") == 0) {
values[i].type = CONFD_COMPLETION_INFO;
values[i].value = "The history must be a non-negative value (preferably 500 or 750)"
i++;
values[i].type = CONFD_COMPLETION;
values[i].value = "500";
values[i].extra = NULL;
i++;
values[i].type = CONFD_COMPLETION;
values[i].value = "750";
values[i].extra = NULL;
i++;
}

if (confd_action_reply_completion(uinfo, values, i) < 0)

268

The CLI agent

confd_fatal("Failed to reply to confd\n");
return CONFD_OK;
}

static int ifs_complete(struct confd_user_info *uinfo, int cli_style,
char *token, int completion_char, confd_hkeypath_t *kp,
char *cmdpath, char *cmdparam_id,
struct confd_qname *simpleType, char *extra) {
char keypath[BUFSIZ] = {0};
struct confd_completion_value values[6];
int i = 0;

if (completion_char == '?') {
values[i].type = CONFD_COMPLETION;
values[i].value = "GigaEthernet0/1";
values[i].extra = "GigabitEtherenet IEEE 802.3z";
i++;
values[i].type = CONFD_COMPLETION;
values[i].value = "GigaEthernet0/2";
values[i].extra = "GigabitEtherenet IEEE 802.3z";
i++;
values[i].type = CONFD_COMPLETION;
values[i].value = "GigaEthernet1/1";
values[i].extra = "GigabitEtherenet IEEE 802.3z";
i++;
values[i].type = CONFD_COMPLETION;
values[i].value = "GigaEthernet1/2";
values[i].extra = "GigabitEtherenet IEEE 802.3z";
i++;
values[i].type = CONFD_COMPLETION;
values[i].value = "FastEthernet0/1";
values[i].extra = "FastEthernet IEEE 802.3";
i++;
}

if (confd_action_reply_completion(uinfo, values, i) < 0)
confd_fatal("Failed to reply to confd\n");
return CONFD_OK;
}

Take a look at the completion callback example which comes bundled with the ConfD distribution, i.e.
cli/completions and read more about the completion callback API in the confd_lib_dp(3) manual
page.

16.14. Using the comment characters # or !
All characters following a # (in J-style) or ! (in C-style) character up to the next newline are ignored. This
makes it possible to have comments in a file containing CLI commands, and still be able to paste the file
into the command-line interface. For example:

269

The CLI agent

# Command file create 2006-05-20 by Joe Smith
# First show the configuration before we change it
show configuration
# Enter configuration mode and add joe as user
configure
wizard adduser
joe
foobar
foobar
admin
commit
exit
# Done

If we want to enter the # or the ! character as an argument, it has to be prefixed with a backslash (\) or
used inside quotes (").

16.15. Annotations and tags
If you have large configurations it may make sense to be able to associate comments (annotations) and
tags with the different parts. And then be able to filter the configuration with respect to the annotations or
tags. For example, tagging parts of the configuration that relates to a certain department or customer.
ConfD has support for both tags and annotations. The support is enabled by enabling configuration
attributes in the confd.conf file. I.e.,
true

Once attributes has been enabled a set of new commands will be available in the CLI for annotating and
tagging parts of the configuration. There will also be a set of pipe targets for controlling whether the tags
and annotations should be displayed, and for filtering depending on annotation and tag content.
The new commands are:
• annotate  
• tag add  
• tag del  
• tag clear  
The annotations and tags will be displayed as comments where the tags are prefixed by Tags:. For example:
joe@io 16:10:17% annotate aaa authentication users user admin "Only allow the XX department
[ok][2009-09-29 16:17:16]
[edit]
joe@io 16:17:16% tag add aaa authentication users user oper foo
[ok][2009-09-29 16:28:28]
[edit]
joe@io 16:29:02% show aaa authentication users user | tags foo
/* Tags: foo */
user oper {
uid
1000;
gid
1000;

270

The CLI agent

password
$1$mfy4jdVt$dNJbiaylcbjpNIeRvHs3X0;
ssh_keydir /var/confd/homes/oper/.ssh;
homedir
/var/confd/homes/oper;
}
[ok][2009-09-29 16:29:18]
[edit]
joe@io 16:29:29% show aaa authentication users user | annotation *XX*
/* Only allow the XX department access to this user. */
user admin {
uid
1000;
gid
1000;
password
$1$Qe$71aKksCOyR.KTBG6ojcGg1;
ssh_keydir /var/confd/homes/admin/.ssh;
homedir
/var/confd/homes/admin;
}
[ok][2009-09-29 16:29:33]
[edit]
joe@io 16:29:33%

It is possible to hide the tags and annotations when viewing the configuration, or to explicitly include them
in the listing. This is done using the display annotations/tags and hide annotations/tags pipe targets.
Note that annotations are tags are part of the configuration. If you add, remove or modify an annotation
or tag you need to commit the new configuration, just as you would if you have made any other change to
the configuration. In I-style this will happen automatically once you press enter.
See the examples.confd/cli/annotations example for a hands on experience.

16.16. Activate and Deactivate
It may be useful to be able to deactivate parts of a configuration without actually removing it from the
configuration file. It makes it possible to, for example, pre-provision a device or temporarily disable parts
of a configuration without loosing the config.
ConfD has support for activate/deactivate. The support is enabled by enabling configuration attributes in
the confd.conf file. I.e.,
true

Once inactive has been enabled two new commands becomes available in configure mode: activate and
deactivate. The argument to the command is a path into the configuration with the same properties as a
path that is deletable.
The new commands are:
• activate 
• deactivate 
A deactivated statement will be indicated by a comment that says Inactive on a line before the deactivated
statement in C- and I-style, and by an inactive: prefix in J-style.
In the J-style CLI
joe@io% deactivate server 1

271

The CLI agent

[ok][2010-11-02 14:45:25]
[edit]
joe@io% show server
inactive: server 1 {
ip
1.1.1.1;
port 1071;
}
server 2 {
ip
1.1.1.2;
port 1072;
}
server 3 {
ip
1.1.1.3;
port 1073;
}
[ok][2010-11-02 14:45:33]
[edit]
joe@io%

In the C- and I-style CLIs
io(config)# deactivate server 1
io(config)# show config
/* Inactive */
server 1
!
io(config)# show configuration merge server
/* Inactive */
server 1
ip
1.1.1.1
port 1071
!
server 2
ip
1.1.1.2
port 1072
!
server 3
ip
1.1.1.3
port 1073
!
io(config)#

See the examples.confd/cli/annotations example for a hands on experience.

16.17. CLI messages
Messages appear when we enter and exit configure mode, when we commit a configuration, and when we
type a command or value that is not valid.
Examples:
admin@io> show c
----------------^
syntax error: unknown command
Possible commands starting with c:
configuration - Display current configuration

272

The CLI agent

cli
- Display cli settings
admin@io> show configuration
------------------^
syntax error: unknown command
[error][2006-06-09 10:12:05]
admin@io% set aaa auth
----------------------^
syntax error: element does not exist
Possible values starting with auth:
authentication
authorization

When we commit a configuration, the CLI first validates the configuration and if there is a problem
indicates what the problem is, for example a missing identifier or a value out of range. A message indicates
where the errors are.
Examples:
admin@io> configure
Entering configuration mode "private"
[ok][2006-06-02 12:31:59]
[edit]
admin@io% set aaa authorization rules rule 200
[ok][2006-06-02 12:31:59]
[edit]
admin@io% commit
Aborted: value is unset 'aaa authorization rules rule 200 action'
[error][2006-06-02 12:31:59]
[edit]
admin@io%

16.18. confd.conf settings
Parts of the CLI behavior can be controlled from the confd.conf file. See the confd.conf.5 man page
for a comprehensive description of all the options.

16.19. CLI Environment
There are a number of session variables in the CLI. They are only used during the session and are not
persistent. Their values are inspected using show cli in operational mode, and set using set in operational
mode. Their initial values are in order derived from the content of the confd.conf file, and the global
defaults as configured under /aaa:session, and finally from user specific settings configured under under /
aaa:user{}/setting.
admin@io> show cli
cli {
autowizard true;
complete-on-space true;
ignore-leading-space false;
history 100;
idle-timeout 1800;
output {
file terminal;

273

The CLI agent

}
paginate true;
screen {
length 82;
width 80;
}
show {
defaults false;
}
terminal xterm;
}
[ok][2006-06-02 12:31:59]
admin@io>

The different values control different parts of the CLI behavior.
autowizard (true | false)
When enabled, the CLI will prompt the user for required settings when a new identifier is created and
for mandatory action parameters.
For example:
admin@io% set aaa authorization rules rule 200
Value for 'context' (): *
Value for 'path' (): *
Value for 'group' (): *
Value for 'op' (): rw
Value for 'action' [reject,accept]: reject
[ok][2006-06-02 12:31:59]
[edit]
admin@io%

This saves the user from typing explicit set commands to set each required setting.
Note that it is recommended to disable the autowizard before pasting in a list of commands, in order
to avoid prompting. A good practice is to start all such scripts with a line that disables the autowizard:
set autowizard false
...
set autowizard true

complete-on-space (true | false)
Controls if command completion should be attempted when  is entered. Entering  always
results in command completion.
devtools (true | false)
Controls if certain commands that are useful for developers should be enabled. The commands xpath
and timecmd are examples of such a command.
ignore-leading--space (true | false)
Controls if leading spaces should be ignored or not. This is useful to turn off when pasting commands
into the CLI.
history ()
Size of CLI command history.
idle-timeout ()
Maximum idle time before being logged out. Use 0 (zero) for for infinity.

274

The CLI agent

paginate (true | false)
Some commands paginate their output, for example. This can be disabled or enabled. It is enabled
by default.
screen width ()
Current width of terminal. This is used when paginating output to get proper line count.
screen length ()
Current length of terminal. This is used when paginating output to get proper line count.
service prompt config
Controls whether a prompt should be displayed in configure mode in the C- and I-style CLI. If set
to false then no prompt will be displayed. The setting is changed using the commands no service
prompt config and service prompt config in configure mode.
show defaults (true | false)
Controls if defaults values should be shown when displaying the configuration. The default values
are shown as comments after the configured value.
For example:
admin@io% show hosts host
host x15 {
domain tail-f.com;
defgw 10.1.5.2;
interfaces {
interface 1 {
name eth0;
ipMask {
ip 192.68.0.60;
# 10.0.0.0
mask 255.255.0.0;
# 255.255.0.0
}
enabled false;
}
}
}
[ok][2006-06-02 12:31:59]
[edit]
admin@io%

terminal (string)
Terminal type. This setting is used for controlling how line editing is performed. Supported terminals
are: dumb, vt100, xterm, linux, and ansi. Other terminals may also work but have no explicit support.

16.20. Commands in J-style
It is possible to get a full XML listing of the commands available in a running ConfD instance by using
the confd option --cli-j-dump . The generated file is only intended for documentation purposes and
cannot be used as input to confdc.

16.20.1. Operational mode commands
compare startup [brief] []
Compare current configuration to the startup configuration. This command is only available when
the system has been configured to have a startup configuration. Differences will be annotated with (removed) and + (added). With the brief option, only the actual diffs will be shown.

275

The CLI agent

compare file  [brief] []
Compare current configuration to a configuration stored on file, i.e. previously saved using the save
command. Differences will be annotated with - (removed) and + (added). With the brief option, only
the actual diffs will be shown.
configure (private | exclusive | shared)
Enter configure mode. The default is private. The options have slightly different meaning depending
on how the system is configured; with a writable running configuration, with a startup configuration,
and with a candidate configuration.
private (writable running enabled)
Edit a private copy of the running configuration, no lock is taken.
private (writable running disabled, startup enabled)
Edit a private copy of the startup configuration, no lock is taken.
exclusive (candidate enabled)
Lock the running configuration and the candidate configuration and edit the candidate
configuration.
exclusive (candidate disabled, startup enabled)
Lock the running configuration (if enabled) and the startup configuration and edit the startup
configuration.
shared (writable running enabled, candidate enabled)
Edit the candidate configuration without locking it.
Example:
admin@io> configure private
Entering configuration mode "private"
[ok][2006-06-02 12:31:59]
[edit]
admin%

file show 
Display contents of a .
Example:
admin@io> file show /etc/skel/.bash_profile
# /etc/skel/.bash_profile
# This file is sourced by bash for login shells. The following line
# runs our .bashrc and is recommended by the bash info pages.
[[ -f ~/.bashrc ]] && . ~/.bashrc
[ok][2006-06-02 12:31:59]
admin@io>

file list 
List files in .
Example:
admin@io> file list /config
rollback0
rollback1
rollback2
rollback3
rollback4

276

The CLI agent

[ok][2006-06-02 12:31:59]
admin@io>

help 
Display help text related to .
Example:
admin@io> help request job
Help for command: request job
Job operations
[ok][2006-06-02 12:31:59]
admin@io>

request  
Invokes the action found at path using the supplied parameters. For example, given the following
action specification in a yang file:
tailf:action shutdown {
tailf:actionpoint actions;
input {
tailf:constant-leaf flags {
type uint64 {
range "1 .. max";
}
tailf:constant-value 42;
}
leaf timeout {
type xs:duration;
default PT60S;
}
leaf message {
type string;
}
container options {
leaf rebootAfterShutdown {
type boolean;
default false;
}
leaf forceFsckAfterReboot {
type boolean;
default false;
}
leaf powerOffAfterShutdown {
type boolean;
default true;
}
}
}
}

the action can be invoked in the following way

admin@io> request shutdown timeout 10s message reboot options { forceFsckAfterReboot tru

request system logout user ( | )
Log out a specific user or session from the device. If the user held the configure exclusive lock, it
will be released.


Log out a specific user.

277

The CLI agent



Terminate a specific session.

Example:
admin@io> show users
SID USER TYPE FROM
PROTO
LOGIN
*4 admin cli 127.0.0.1 console 13:11:03
3 oper cli 127.0.0.1 console 13:11:01
[ok][2006-06-02 12:31:59]
admin@io> request system logout user oper
[ok][2006-06-02 12:31:59]
admin@io> show users
SID USER TYPE FROM
PROTO
LOGIN
*4 admin cli 127.0.0.1 console 13:11:03
[ok][2006-06-02 12:31:59]
admin@io>

request message (all | ) 
Display a message on the screens of all users who are logged in to the device or on a specific screen.
all
Display the message to all currently logged in users.

Display the message to a specific user.
Example:
admin@io> request message oper "I will reboot system in 5 minutes."
admin@io>
oper@io> Message from admin@io at 13:16:41...
I will reboot system in 5 minutes.
EOF

request job stop 
Stop a specific background job. In the default CLI the only command that creates background jobs
is monitor start.
Example:
admin@io> monitor start /var/log/messages
[ok][2006-06-02 12:31:59]
admin@io> show jobs
JOB COMMAND
3
monitor start /var/log/messages
[ok][2006-06-02 12:31:59]
admin@io> request job stop 3
[ok][2006-06-02 12:31:59]
admin@io> show jobs
JOB COMMAND
[ok][2006-06-02 12:31:59]
admin@io>

set (complete-on-space | ignore-leading-space | idle-timeout | paginate | screen length | screen width
| terminal |autowizard | show defaults) 
Set CLI properties.
Example:

278

The CLI agent

admin@io> set autowizard true
[ok][2006-06-02 12:31:59]
admin@io>

set-path 
This commands lets you 'cd' into a status part of the tree. Similar to the 'edit' command in configure
mode.
show cli
Display CLI properties.
Example:
admin@io> show cli
cli {
autowizard true;
complete-on-space true;
ignore-leading-space false;
history 100;
idle-timeout 1800;
output {
file terminal;
}
paginate true;
screen {
length 82;
width 80;
}
show {
defaults false;
}
terminal xterm;
}
[ok][2006-06-02 12:31:59]
admin@io>

show cli history []
Display CLI command history. By default the last 100 commands are listed. The size of the history
list is configured using the history CLI setting. If you specify a history limit, only the last number of
commands up to that limit will be shown.
Example:
admin@io> show cli history
06-19 14:34:02 -- ping router
06-20 14:42:35 -- show configuration
06-20 14:42:37 -- show users
06-20 14:42:40 -- show cli history
[ok][2006-06-20 14:42:40]
admin@io> show cli history 3
14:42:37 -- show users
14:42:40 -- show cli history
14:42:46 -- show cli history 3
[ok][2006-06-20 14:42:46]
admin@io>

show all [details] []
Display both configuration and status. By default the whole configuration and the whole status tree is
displayed. It is possible to limit what is shown by supplying a pathfilter.

279

The CLI agent

The pathfilter may be either a path pointing to a specific instance, or if an instance id is omitted, the
part following the omitted instance is treated as a filter.
show configuration [details] []
Display current configuration. By default the whole configuration is displayed. It is possible to limit
what is shown by supplying a pathfilter.
The pathfilter may be either a path pointing to a specific instance, or if an instance id is omitted, the
part following the omitted instance is treated as a filter.
Example:
To show the aaa settings for the admin user, you can do:
admin@io> show configuration aaa authentication users user admin
uid 100;
gid 10;
password $1$feedbabe$nGlMYlZpQ0bzenyFOQI3L1;
ssh_keydir /var/confd/homes/admin/.ssh;
homedir /var/confd/homes/admin;
[ok][2006-07-31 17:16:01]
admin@io>

To show all users that have group id 10, you would omit the user id, and instead specify gid 10.
admin@io> show configuration aaa authentication users user gid 10
user admin {
uid 100;
gid 10;
password $1$feedbabe$nGlMYlZpQ0bzenyFOQI3L1;
ssh_keydir /var/confd/homes/admin/.ssh;
homedir /var/confd/homes/admin;
}
user oper {
uid 100;
gid 10;
password $1$feedbabe$i2glnaB.iUj2VXh/zlq.o/;
ssh_keydir /var/confd/homes/oper/.ssh;
homedir /var/confd/homes/oper;
}
[ok][2006-07-31 17:16:56]
admin@io>

show parser dump 
Shows all possible commands starting with command prefix.
show status 
Display current values of read-only nodes in the configuration.
show table 
This command shows the configuration as a table provided that path leads to a list element.
Example:
admin@io 09:42:08> show table aaa authentication groups group
NAME
GID USERS
--------------------------admin
admin private
oper
oper public

280

The CLI agent

[ok][2007-09-17 09:44:12]
[edit]
admin@io 09:44:12>

show users
Display currently logged on users. The current session, i.e. the session running the show status
command, is marked with an asterisk.
Example:
admin@io> show users
SID USER TYPE FROM
PROTO
LOGIN
*4 admin cli 127.0.0.1 console 13:11:03
3 oper cli 127.0.0.1 console 13:11:01
[ok][2006-06-02 12:31:59]
admin@io>

show jobs
Display currently running background jobs.
Example:
admin@io> show jobs
JOB COMMAND
3
monitor start /var/log/messages

show log 
Display contents of a log file.
Example:
admin@io> show log messages

source 
Execute commands from  as if they had been entered by the user. The autowizard is disabled
when executing commands from the file.
commit (abort | confirm) [persist-id ]
Abort or confirm a pending confirming commit. A pending confirming commit will also be aborted
if the CLI session is terminated without doing commit confirm. The default is confirm.
If the confirming commit has been initiated with the persist option then user needs to supply the same
token as a persist-id for the commit to have effect.
Example:
admin@io> commit abort

describe 
Display detailed information about a command.
Example:
admin@io> describe ping
Common
Source : clispec
File
: commands-c.cli

281

The CLI agent

Callback [os command]
OS command : ping
Run as user : confd
Help
Verify IP (ICMP) connectivity to a host.
Info
Ping a host

timecmd 
Time command. It measures and displays the execution time of .
Note that this command will only be available if devtools has been set to true in the CLI session
settings.
Example:
io# timecmd id
user = admin(501), gid=20, groups=admin, gids=12,20,33,61,79,80,81,98,100
Command executed in 0.00 sec
[ok][2016-09-16 14:06:15]
io#

16.20.2. Configure mode commands
activate 
Activate a statement that has previously been deactivated.
Only available in when the system has been configured with support for inactive.
deactivate 
Deactivate a statement in the configuration.
Only available in when the system has been configured with support for inactive.
annotate  
Associate an annotation with a given configuration statement. To remove an annotation leave the text
empty.
Only available in when the system has been configured with attributes enabled.
commit (check | and-quit | confirmed [] [persist ] | to-startup) [comment ]
[label ] [persist-id ]
Commit current configuration to running.
check
Validate current configuration.
and-quit
Commit to running and quit configure mode.
to-startup
Commit current configuration to running and startup configuration. Only available if the system
is configured to have a startup configuration.

282

The CLI agent

confirmed
Commits the current configuration to running with a timeout. If no commit confirm command
has been issued before the timeout expires, then the configuration will be reverted to the
configuration that was active before the commit confirmed command was issued. If no timeout
is given then the confirming commit will have a timeout of 10 minutes. The configuration session
will be terminated after this command since no further editing is possible. Only available in
configure exclusive and configure shared mode when the system has been configured with a
candidate.
The confirming commit will be rolled back if the CLI session is terminated before confirming
the commit, unless the persist argument is given. If the persist command is given then the CLI
session can be terminated and a later session may confirm the pending commit by supplying the
persist token as an argument to the commit command using the persist-id argument.
comment 
Associate a comment with the commit. The comment can later be seen when examining rollback
files.
label 
Associate a label with the commit. The label can later be seen when examining rollback files.
persist-id 
If a prior confirming commit operation has been performed with the persist argument, then to
modify the ongoing confirming commit process the persist-id argument needs to be supplied with
the same persist token. This makes it possible to, for example, abort an onging persist commit,
or extend the timeout.
validate
Validates current configuration. This is the same operation as commit check.
insert 
Inserts a new element. If the element already exists and has the indexedView option set in the data
model, then the old element will be renamed to element+1 and the new element inserted in its place.
insert  [first | last| before key| after key]
Insert a new element into an ordered list. The element can be added first, last (default), before or after
another element.
move [first|last|beforekey|afterkey]
Move an existing element to a new position in an ordered list. The element can be moved first, last
(default), before or after another element.
rename  
Rename an instance.
delete 
Delete a data element.
edit 
Edit a sub-element. Missing elements in path will be created.
exit (level | configuration-mode)
level

Exit from this level. If performed on the top level, will exit configure
mode. This is the default if no option is given.

configuration-mode

Exit from configuration mode regardless of which edit level.

283

The CLI agent

help 
Shows help text for command.
hide 
Re-hides the elements and actions belonging to the hide groups. No password is required for hiding.
Note that this command is hidden and not shown during command completion.
unhide 
Unhides all elements and actions belonging to the hide-group. You may be required to enter a
password. Note that this command is hidden and not shown during command completion
load (merge | replace | override) ( | terminal)
Load configuration from file or terminal.
merge

Merge content of file/terminal with current configuration.

replace

Replace the content of file/terminal for the corresponding parts of the current
configuration. This is different from override in the that only the parts that occur
in the file/terminal are replace, the rest of the configuration is left as is. In the case
of override the entire configuration is deleted (with the exception of hidden data)
before loading the new configuration from the file/terminal.
There currently exists a discrepancy in behavior between different CLI:s and file
formats. List nodes will be replaced for the following combinations:
• Juniper CLI, XML and curly bracket format.
• Cisco CLI, XML format.
List nodes will be merged for the following combination:
• Cisco CLI, curly bracket format.

override

The current configuration is deleted and a new configuration is loaded from file/
terminal. Hidden data is not affected.

The configuration file may contain replace: and delete: directives. For example if you have the
configuration
system {
parent-mo {
child-mo
attr
}
child-mo
attr
}
}
}

1 {
10;
2 {
5;

and want to delete child-mo 2, you can create a configuration file containing either (using replace:)
system {
replace:
parent-mo {
child-mo 1 {
attr 2;
}
}
}

284

The CLI agent

or (using delete:)
system {
parent-mo {
delete:
child-mo 2 {
attr 5;
}
}
}

save  [xml] []
Save the whole or parts of the current configuration to file. By the default the configuration is saved
in curly bracket format. If the xml argument is given then the configuration is saved in XML format.
rollback []
Return the configuration to a previously committed configuration. The system stores a limited number
of old configurations. The number of old configurations to store is configured in the confd.conf
file. If more than the configured number of configurations are stored, then the oldest configuration
is removed before creating a new one.
The most recently committed configuration (the running configuration) is number 0, the next most
recent 1, etc.
The files are called rollback0 - rollbackX, where X is the maximum number of saved committed
configurations.
Example:
admin@io% rollback 1
[ok][2006-06-02 12:31:59]
admin@io%

Note that this command is only available if rollback has been enabled in confd.conf.
run 
Run command in operational mode.
set  []
Set a parameter. If a new identifier is created and autowizard is enabled, then the CLI will prompt
the user for all mandatory sub-elements of that identifier. It will also prompt for mandatory action
parameters.
If no  is provided, then the CLI will prompt the user for the value. No echo of the entered
value will occur if  is an encrypted value, i.e. of the type tailf:md5-digest-string, tailf:des3cbc-encrypted-string, or tailf:aes-cfb-128-encrypted-string as described in confd_types(3).
show [details] []
Show current configuration. The show command can be limited to a part of the configuration by
providing a .
show parser dump 
Shows all possible commands starting with command prefix.
compare running [brief] []
Compare current configuration to the running configuration. Differences will be annotated with (removed) and + (added). With the brief option, only the actual diffs will be shown.

285

The CLI agent

compare startup [brief] []
Compare current configuration to the startup configuration. This command is only available when the
system has been configured to have a startup configuration. With the brief option, only the actual
diffs will be shown.
compare file  [brief] []
Compare current configuration to a configuration stored on file, i.e. previously saved using the save
command. With the brief option, only the actual diffs will be shown.
tag add  
Add a tag to a configuration statement.
Only available in when the system has been configured with attributes enabled.
tag del  
Remove a tag from a configuration statement.
Only available in when the system has been configured with attributes enabled.
tag clear 
Remove all tags from a configuration statement.
Only available in when the system has been configured with attributes enabled.
top [command]
Exit to top level of configuration, or execute a command at the top level of the configuration.
up [command]
Exit one level of configuration, or execute a command at one level up.
revert
Copy running configuration into current configuration.
status
Display users currently editing the configuration.
describe [ | ]
Display detailed information about a command. This information may for example consist of the
source of the command (YANG, clispec or built-in), the corresponding path in the YANG file (in case
of an auto-rendered command) and information regarding what callpoints, actionpoints and validation
points that may be tied to the command. Due to the verbose information that may be displayed, it may
be desirable to restrict the usage of this command by including it in an appropriate set of authorization
rules or by the means of any other authorization functionality.
Example:
admin@io% describe dhcp
Common
Source
: YANG
Module
: dhcpd
Namespace
: http://tail-f.com/ns/example/dhcpd
Path
: /dhcp
Node
: container
Exported agents : all
Checksum
: 3c893927631cceee3700c23bb38cd050

286

The CLI agent

xpath [ctx ] (eval | must | when) 
Evaluate an XPath expression. A context-path may be given to be used as the current context for
the evaluation of the expression. If no context-path is given, the current sub-mode will be used as
the context-path. The pipe command trace may be used to display debug/trace information during
execution of the command.
Note that this command will only be available if devtools has been set to true in the CLI session
settings.
eval
Evaluate an XPath expression.
must
Evaluate the expression as a YANG must expression.
when
Evaluate the expression as a YANG when expression.
timecmd 
Time command. It measures and displays the execution time of .
Note that this command will only be available if devtools has been set to true in the CLI session
settings.
Example:
io# timecmd status
Users currently editing the configuration:
admin console (cli from 127.0.0.1) on since 2016-09-16 14:15:29 private mode
Command executed in 0.00 sec
[ok][2016-09-16 14:16:01]
[edit]
io#

16.21. Commands in C/I-style
It is possible to get a full XML listing of the commands available in a running ConfD instance by using
the confd option --cli-c-dump , for C-style and --cli-i-dump for I-style. The generated file is only
intended for documentation purposes and cannot be used as input to confdc.

16.21.1. Operational mode commands
The IOS CLI does not have any of the commands associated with transactions, i.e. commit, abort, show
configuration.
The IOS CLI has commands for entering and leaving privileged mode, settings passwords and secrets and
assigning privilege levels to commands in EXEC mode.
The privilege information for the IOS mode is stored in the AAA namespace under the aaa/ios element.
Note that all EXEC commands are available at level 15 without needing to specify them in the aaaconfiguration.
It is possible to assign custom prompt to the different levels in EXEC mode. The default is for level 0 to
have the "\h> " prompt and for all other levels to have the "\h# " prompt.

287

The CLI agent

compare startup [brief] []
Compare current configuration to the startup configuration. This command is only available when
the system has been configured to have a startup configuration. Differences will be annotated with (removed) and + (added). With the brief option, only the actual diffs will be shown.
compare file  [brief] []
Compare current configuration to a configuration stored on file, i.e. previously saved using the save
command. Differences will be annotated with - (removed) and + (added). With the brief option, only
the actual diffs will be shown.
config (terminal | shared | exclusive )
Enter configure mode. The default is terminal. The options have slightly different meaning depending
on how the system is configured; with a writable running configuration, with a startup configuration,
and with a candidate configuration.
terminal (writable running enabled)
Edit a private copy of the running configuration, no lock is taken.
terminal (writable running disabled, startup enabled)
Edit a private copy of the startup configuration, no lock is taken.
exclusive (candidate enabled)
Lock the running configuration and the candidate configuration and edit the candidate
configuration.
exclusive (candidate disabled, startup enabled)
Lock the running configuration (if enabled) and the startup configuration and edit the startup
configuration.
shared (writable running enabled, candidate enabled)
Edit the candidate configuration without locking it.
Example:
io# config terminal
Entering configuration mode terminal
io(config)#

enable ()
Only available in IOS (i) mode. Enables privileged EXEC commands. The default level is 15. The
CLI will prompt for a password if a password has been assigned to the level.
Example:
io> enable
io#

disable ()
Only available in IOS (i) mode. Downgrade to a lower privilege level.
Example:
io# disable 4
io#

file show 
Display contents of a .
Example:

288

The CLI agent

io# file show /etc/skel/.bash_profile
# /etc/skel/.bash_profile
# This file is sourced by bash for login shells. The following line
# runs your .bashrc and is recommended by the bash info pages.
[[ -f ~/.bashrc ]] && . ~/.bashrc
io#

file list 
List files in .
Example:
io# file list /config
rollback0
rollback1
rollback2
rollback3
rollback4
io#

help 
Display help text related to .
Example:
io# help config
Help for command: config
Manipulate software configuration information
io#

id
Show user id information; uid, gid, and groups
Example:
io# id
user = joe(1000), gid=100, groups=wheel, gids=10,100
io#

message (all | ) 
Display a message on the screens of all users who are logged in to the device or on a specific screen.
all
Display the message to all currently logged in users.

Display the message to a specific user.
Example:
io# message all "I will reboot the system in 5 minutes."
Message from joe@io at 13:26:49...
I will reboot the system in 5 minutes.
EOF
io#

job stop 
Stop a specific background job. In the default C-style CLI there are no commands that create
background jobs. Custom commands can be created that do this.

289

The CLI agent

Example:
io#
JOB
1
io#
io#
JOB
io#

show jobs
COMMAND
monitor start /tmp/saved
job stop 1
show jobs
COMMAND

show cli
Display CLI properties.
Example:
io# show cli
autowizard
complete-on-space
ignore-leading-space
history
idle-timeout
output-file
paginate
screen-length
screen-width
show-defaults
terminal
io#

true
true
false
50
1800
terminal
true
82
80
false
xterm

history []
Display CLI command history. By default the last 100 commands are listed. The size of the history
list is configured using the CLI history setting. If you specify a history limit, only the last number of
commands up to that limit will be shown.
Example:
io# history
13:28:46 -13:28:53 -13:29:05 -13:29:51 -13:29:53 -13:30:31 -io#

show jobs
job stop 1
show jobs
show
show cli
history

show configuration commit list 
List rollback files (C-style only). Note that this command is only available if rollback has been enabled
in confd.conf.
show notification stream  [last ] [from ] [to ]
Display the last notifications in a selected stream. The stream must use the builtin store and have
replay enabled. It is possible to limit the output by specifying the maximum number of events and/
or a time range.
show parser dump 
Shows all possible commands starting with command prefix.

290

The CLI agent

show running-config [details | all] []
Display current configuration. By default the whole configuration is displayed. It is possible to limit
what is shown by supplying a pathfilter. The pathfilter may be either a path pointing to a specific
instance, or if an instance id is omitted, the part following the omitted instance is treated as a filter.
Example:
To show the aaa settings for the admin user, you can do:
io# show running-config aaa authentication users user admin
aaa authentication users user admin
uid
1000
gid
100
password
$1$fB$0w68PmacQ4VmE3/M3nK3Ug==
ssh_keydir /var/confd/homes/admin/.ssh
homedir
/var/confd/homes/admin
!
io#

To show all users that have group id 10, you would omit the user id, and instead specify gid 10.
io# show running-config aaa authentication users user gid 100
aaa authentication users user admin
uid
1000
gid
100
password
$1$fB$0w68PmacQ4VmE3/M3nK3Ug==
ssh_keydir /var/confd/homes/admin/.ssh
homedir
/var/confd/homes/admin
!
aaa authentication users user oper
uid
1000
gid
100
password
$1$S6$brGZW9wSDifHoU7Rf5KSHA==
ssh_keydir /var/confd/homes/oper/.ssh
homedir
/var/confd/homes/oper
!

Per default only elements that have been explicitly set to a value are shown. This makes it easier to
handle large configurations. However, it is possible to force the show command to display all elements.
This is done using the 'details' or 'all' options.
show startup-config [details | all] []
Display the startup configuration. This command is only available if ConfD has been configured with
a startup configuration. By default the whole configuration is displayed. It is possible to limit what is
shown by supplying a pathfilter. The pathfilter may be either a path pointing to a specific instance, or
if an instance id is omitted, the part following the omitted instance is treated as a filter.
Per default only elements that have been explicitly set to a value are shown. This makes it easier to
handle large configurations. However, it is possible to force the show command to display all elements.
This is done using the 'details' or 'all' options.
write terminal []
Display current configuration.
copy running-config startup-config
Copy running configuration to startup configuration. Only available when the system has been
configured to have a startup database.

291

The CLI agent

write memory
Copy running configuration to startup configuration. Only available when the system has been
configured to have a startup database.
source 
Execute commands from  as if they had been entered by the user. The autowizard is disabled
when executing commands from the file.
show 
Display current values of read-only parameters. If a list element is encountered then the command
attempts to arrange the output as a table.
who
Display currently logged on users. The current session, i.e. the session running the show status
command, is marked with an asterisk.
Example:
io# who
Session User Context From
Proto
Date
*7
joe cli
127.0.0.1 console 13:19:05
io#

logout ( | )
Terminates the current session.
logout ( | )
Log out a specific user or session from the device. If the user held the configure exclusive lock, it
will be released.


Log out a specific user.



Terminate a specific session.

Example:
io# who
Session User Context From
Proto
Date
*5
admin cli
127.0.0.1 console 10:25:46
4
jb
cli
127.0.0.1 console 10:25:37
io# logout jb
io# who
Session User Context From
Proto
Date
*5
admin cli
127.0.0.1 console 10:25:46
io#

show jobs
Display currently running background jobs.
Example:
io# show jobs
JOB COMMAND
2
monitor start /tmp/saved
io#

show log 
Display contents of a log file.

292

The CLI agent

Example:
io# show log messages

commit (abort | confirm) [persist-id ]
Abort or confirm a pending confirming commit. A pending confirming commit will also be aborted
if the CLI session is terminated without doing commit confirm. The default is confirm.
If the confirming commit was initiated with a persist argument then the same token needs to be
supplied using the persist-id argument to this command.
Example:
io# commit abort

timestamp (enable | disable)
Display a timestamp after a command has been executed. The timestamp is displayed in the timezone
UTC+-00:00 by default. A UTC offset may be configured in confd.conf.
Example:
io# timestamp enable
io# config
Tue Mar 12 11:31:03.698 UTC
Entering configuration mode terminal
io(config)#

describe 
Display detailed information about a command.
Example:
# describe ping
Common
Source : clispec
File
: commands-c.cli
Callback [os command]
OS command : ping
Run as user : confd
Help
Verify IP (ICMP) connectivity to a host.
Info
Ping a host

timecmd 
Time command. It measures and displays the execution time of .
Note that this command will only be available if devtools has been set to true in the CLI session
settings.
Example:
io# timecmd pwd
At top level

293

The CLI agent

Command executed in 0.00 sec
io#

16.21.2. Configure mode commands
alias (  )
Defines the alias alias-name. It will be expanded to alias-expansion.
It is possible to define parametrised aliases, i.e. aliases that accepts parameters. The parameters are
then expanded when the alias is applied.
The alias can be used anywhere on the command line. After a command with an alias has been entered,
the expanded command line is displayed so that you can verify the alias value.
For example:
io(config)#
io(config)#
io(config)#
io(config)#
io(config)#
...
io(config)#
io(config)#

alias foo(a,b) "show $(a) ; show $(b)"
alias myUser c87923
commit
foo(history,configuration)
show history ; show configuration
aaa authentication users user myUser
aaa authentication users user c87923

The aliases are stored persistently in cdb in the aaa namespace. Note that the clispec file needs to
contain the following three entries in the modifications section:




activate 
Activate a statement in the configuration that has previously been deactivated.
Only available in when the system has been configured with support for inactive.
deactivate 
Deactivate a statement in the configuration.
Only available in when the system has been configured with support for inactive.
annotate  
Associate an annotation with a given configuration statement. To remove an annotation leave the text
empty.
Only available in when the system has been configured with attributes enabled.
tag add  
Add a tag to a configuration statement.
Only available in when the system has been configured with attributes enabled.
tag del  
Remove a tag from a configuration statement.

294

The CLI agent

Only available in when the system has been configured with attributes enabled.
tag clear 
Remove all tags from a configuration statement.
Only available in when the system has been configured with attributes enabled.
enable (secret | password) level  ( 0 | 7) 
Only available in IOS (i) mode. Configures a password for a specific EXEC level. If both a password
and a secret is configured, the secret is used.
secret | password

Specifies how the password is to be encrypted



The EXEC level to password protect

0|7

Use 0 to indicate that the password given at the end of the command is in
plain text, and 7 for an already encrypted password.



The actual password

Example:
io(config)# enable secret level 3 0 bluebox
io(config)#

privilege  level  
Only available in IOS (i) mode. Configures for which level a command should be available.


In which mode is the command.



In which privilege level should the command be available



Command string.

Example:
io(config)# privilege exec level 4 show
io(config)#

hide 
Re-hides the elements and actions belonging to the hide groups. No password is required for hiding.
Note that this command is hidden and not shown during command completion.
unhide 
Unhides all elements and actions belonging to the hide-group. You may be required to enter a
password. Note that this command is hidden and not shown during command completion
commit (check | and-quit | confirmed [] [persist ] to-startup) [comment ]
[label ] [persist-id ]
Commit current configuration to running.
check
Validate current configuration.
and-quit
Commit to running and quit configure mode.

295

The CLI agent

to-startup
Commit current configuration to running and startup configuration. Only available if the system
is configured to have a startup configuration.
confirmed
Commits the current configuration to running with a timeout. If no commit confirm command
has been issued before the timeout expires, then the configuration will be reverted to the
configuration that was active before the commit confirmed command was issued. If no timeout
is given then the confirming commit will have a timeout of 10 minutes. The configuration session
will be terminated after this command since no further editing is possible. Only available in
configure exclusive and configure shared mode when the system has been configured with a
candidate.
The confirming commit will be rolled back if the CLI session is terminated before confirming
the commit, unless the persist argument is given. If the persist command is given then the CLI
session can be terminated and a later session may confirm the pending commit by supplying the
persist token as an argument to the commit command using the persist-id argument.
comment 
Associate a comment with the commit. The comment can later be seen when examining rollback
files.
label 
Associate a label with the commit. The label can later be seen when examining rollback files.
persist-id 
If a prior confirming commit operation has been performed with the persist argument, then to
modify the ongoing confirming commit process the persist-id argument needs to be supplied with
the same persist token. This makes it possible to, for example, abort an onging persist commit,
or extend the timeout.
validate
Validates current configuration. This is the same operation as commit check.
insert 
Inserts a new element. If the element already exists and has the indexedView option set in the data
model, then the old element will be renamed to element+1 and the new element inserted in its place.
insert [first|last|beforekey|afterkey]
Insert a new element into an ordered list. The element can be added first, last (default), before or after
another element.
move [first|last|beforekey|afterkey]
Move an existing element to a new position in an ordered list. The element can be moved first, last
(default), before or after another element.
rename  
Rename an instance.
no 
Delete or unsets a data element.
exit (level | configuration-mode)
level

Exit from current mode. If performed on the top level, will exit configure
mode. This is the default if no option is given.

296

The CLI agent

configuration-mode

Exit from configuration mode regardless of mode.

help 
Shows help text for command.
load ( )
Load configuration from file.
pwd
Display current submode path.
save  [xml] []
Save the whole or parts of the current configuration to file.
rollback configuration []
Return the configuration to a previously committed configuration. The system stores a limited number
of old configurations. The number of old configurations to store is configured in the confd.conf file. If
more than the configured number of configurations are stored, then the oldest configuration is removed
before creating a new one.
The most recently committed configuration (the running configuration) is number 0, the next most
recent 1, etc.
The files are called rollback0 - rollbackX, where X is the maximum number of saved committed
configurations.
Example:
io(config)# rollback configuration 1
io#

Note that this command is only available if rollback has been enabled in confd.conf.
do 
Run command in operational mode.
show configuration []
Show current configuration buffer. The show command can be limited to a part of the configuration
by providing a .
show configuration diff []
Show configuration changes in diff style, ie new lines prefixed with a plus (+) sign, and removed lines
prefixed with a minus (-) sign.. The show command can be limited .
show configuration commit changes 
Show changes that were committed for a given commit id.
show configuration commit list 
List rollback files (C-style only). Note that this command is only available if rollback has been enabled
in confd.conf.
show configuration rollback changes 
Show changes for rolling back to rollback file nr.
show full-configuration [details] []
Show current configuration. The show command can be limited to a part of the configuration by
providing a .

297

The CLI agent

show parser dump 
Shows all possible commands starting with command prefix.
revert
Copy running configuration into current configuration.
clear
Remove all configuration changes.
describe [ | ]
Display detailed information about a command. This information may for example consist of the
source of the command (YANG, clispec or built-in), the corresponding path in the YANG file (in case
of an auto-rendered command) and information regarding what callpoints, actionpoints and validation
points that may be tied to the command. Due to the verbose information that may be displayed, it may
be desirable to restrict the usage of this command by including it in an appropriate set of authorization
rules or by the means of any other authorization functionality.
Example:
(config)# describe dhcp
Common
Source
: YANG
Module
: dhcpd
Namespace
: http://tail-f.com/ns/example/dhcpd
Path
: /dhcp
Node
: container
Exported agents : all
Checksum
: 3c893927631cceee3700c23bb38cd050

xpath [ctx ] (eval | must | when) 
Evaluate an XPath expression. A context-path may be given to be used as the current context for
the evaluation of the expression. If no context-path is given, the current sub-mode will be used as
the context-path. The pipe command trace may be used to display debug/trace information during
execution of the command
Note that this command will only be available if devtools has been set to true in the CLI session
settings.
eval
Evaluate an XPath expression.
must
Evaluate the expression as a YANG must expression.
when
Evaluate the expression as a YANG when expression.
timecmd 
Time command. It measures and displays the execution time of .
Note that this command will only be available if devtools has been set to true in the CLI session
settings.
Example:
io# timecmd pwd
At top level

298

The CLI agent

Command executed in 0.00 sec
io#

16.22. Customizing the CLI
16.22.1. Modifying builtin commands
There are a number of built-in commands in each CLI style. These can be modified in a number of ways.
It is possible to remove them, rename them, hide them, change their help and info texts, add a command
timeout and to add confirmation prompts. This is done using the  section of the clispec
file. For example:


Are your really sure you know what you are doing?




Enter private configuration mode.

Hi there info configure!
Hi there help configure!

10
 -->


See the clispec.5 man page for a detailed description of each modifications option.

16.22.2. Adding new commands
New commands can be added in two different ways, either as an action in the YANG file, or as a command
in the clispec file. The advantage of using an action is that the command will be available through
all northbound interfaces. However, a clispec command may give you better control over your input
parameters.
It is also possible to use a dedicated data model for the CLI, i.e. a YANG file that is only visible (exported)
in the CLI. This is useful if you want to add custom modes in the C- and I-style CLIs.

16.22.3. Suppressing automatically generated modes
The C- and I- style CLIs will automatically create new modes for all list elements in the configuration,
i.e. all elements that have maxOccurs larger than 1.
It is possible to suppress this behavior by adding a suppressMode direction in the modifications section
of the clispec file.
For example:




The src attribute is the clispec path as it appears in the CLI. It should be the path to a list element.

299

The CLI agent

16.22.4. Creating new configuration modes
Similarly to suppressing an automatically generated mode it is sometimes desirable to create modes at
points in the configuration where there isn't a list element. This can be done trough a declaration in the
modifications section of the clispec file.
For example:




The src attribute should be a path to a static/internal element, i.e. an element with minOccurs="1"
maxOccurs="1".

16.22.5. Custom mode names
ConfD automatically assigns a mode name based on the element name and the key elements of the
container. There are two styles, full and short, where the short style only contains the last element of the
path and the full all components of the path. Which style to use is configured in the confd.conf file.
The automatic mode name can be overridden either by a fixed mode name or by a dynamically generated
mode name. It is done through a modeName declaration in the modifications section of the clispec file.
For example:


hosts


custommode



ConfD will invoke the cmdpoint/action custommode the first time the mode is entered for a given instance.
It will then cache the mode name. The callback will receive the path to the instance as argv/argc arguments
and is expected to reply by calling the function confd_action_reply_command().
An example from actions.c in the cli/c_cli example:
static int do_modename(struct confd_user_info *uinfo,
char *path, int argc, char **argv)
{
int i;
int sock;
char *mode = "config-priv";
printf("do_modename called\n");
printf("path: %s\n", path);
for (i = 0; i < argc; i++) {
printf("param %2d: %s\n", i, argv[i]);
}
if (confd_action_reply_command(uinfo, &mode, 1) < 0)
confd_fatal("Failed to reply to confd\n");

300

The CLI agent

return CONFD_OK;
}

16.22.6. Adding custom show output
It is possible to override the default output from the auto-rendered show commands but not for the J-style
'show configuration', 'show status', 'show all', and 'show table'. It can be done in two different ways. Either
using a show element in the clispec file or as a regular CLI command.
For example:



./aaa_auth.sh




The aaa_auth.sh executable will be invoked whenever the user enters "show aaa authentication users user"
or a path prefix. The command will receive the indent depth and path as arguments. The callback can be
either a C-function (capi), or an executable, and it will be invoked as an ordinary CLI command.
It is also possible to create an ordinary CLI command and mount it on the same path as the show command
target.
For example:





./aaa_group.sh




The advantage with this approach is that the command may take additional parameters, for example 'details'
or 'statistics'.

16.22.7. Command parameters
A custom CLI command can be defined to have a set of parameters. These parameters can be arranged
as a mix of:
• a straight list of command options
• a parameter tree
• a choice list of parameters, where a minimum and maximum number of required parameters can be
specified
Suppose you want a command that accepts a number of options in an arbitrary order. For example:
show alarm [type ] [severity ] [acknowledged] [active]

The parameters can be entered in arbitrary order and they are all optional. This can be achieved with a
choice params list with min and max set.

301

The CLI agent




type
--type 



severity
major minor
--severity 



acknowledged
--acknowledged



active
--active



Suppose instead that you want the following command:
show alarm [type  severity ] [acknowledged] [active]

That is, if you enter type you must also enter severity. You can specify this with a parameter tree where
severity is a child to the type parameter.



type
--type 



severity
major minor
--severity 





acknowledged
--acknowledged



active
--active



You can have nested params lists where some are of choice mode and others are straight lists. However,
in a choice list there cannot be two possible paths given a token, i.e., you cannot have two items that have
the same name, or one named item and one that accepts a generic value. All paths must be deterministic.

302

The CLI agent

For example, you cannot have the following command parameters:



--type 



major minor
--severity 



The reason is that if you enter the parameter major, then the CLI parser cannot determine if it is intended
as the first parameter or the second. The first parameter accepts any input.

16.22.8. Hiding parts of the configuration
It is possible to hide parts of the configuration using the hidden attribute in the yang files, and the hideGroup
attribute in the clispec file. All elements with the same hidden attribute belong to the same 'hide group'
For example (yang):
leaf resetuser {
tailf:hidden debug;
type boolean;
default false;
}

For example (clispec):






test.sh




debug



In the yang example the resetuser leaf belongs to the debug hide group. All hidden yang elements must have
default values or be optional since they cannot be configured by the user unless they have been unhidden.
Elements hidden in this way will be hidden to users when using the Web UI or the CLI, but can optionally
be made visible, i.e. unhidden, through a hidden CLI command. An entry in the confd.conf file is needed
to make this possible.
For example:

debug
verysecret


303

The CLI agent

The debug hide group can be 'unhidden' in the CLI provided that the user knows the name of the hide group
and the password. It is possible to leave out the password parameter in which case the CLI will not prompt
for one. The unhide and hide commands are used to interactively hide and unhide hide groups in the CLI.
It is also possible to define a C-callback that is used to authenticate the user. This is useful when you want
short lived 'unhide' password, or user-specific unhide password.
Note that CLI commands and actions can also be hidden in the same way, but not individual action
parameters. Also, hidden elements are only hidden from the CLI and the Web UI unless the special hide
group full is used, in which case it is hidden from all northbound interfaces as well as the rollback file.
The special hide groups full is useful when the data is only used by the application and should not be part
of the actual configuration of the device.

16.22.9. EXEC commands
The I-style style has an additional concept of privilege levels. Only a subset of commands are initially
available when the user logs on to the box. It is then possible to enable a higher privilege level using the
enable command. Entering a new privilege level may require a password and the prompt may change as
a result of entering the level.
The AAA data model contains a section for controlling which commands are available in the different
levels, which prompt to use for each level and if a password or secret is configured.
The configuration mode commands enable and privilege are used for dynamically modifying this
configuration. The initial configuration should be supplied in the aaa_init.xml file.

16.22.10. File access
The default behavior is to enforce Unix style access restrictions. That is, the users uid, gid, and gids are
used to control what the user has read and write access to.
However, it is also possible to jail a CLI user to its home directory (or the directory where confd_cli is
started). This is controlled using the confd.conf parameter /confdConfig/cli/restrictedFileAccess. If this is
set to true, then the user only has access to the home directory, or if a directory is specified in a cli command
parameter (params/param/type/directory{wd} or params/param/type/file{wd}) to that directory.

16.22.11. Help texts
Help and information texts are specified in a number of places. In the yang files the tailf:info element is
used to specify a descriptive text that is shown when the user enters ? in the CLI. The first sentence of the
description text is used when showing one-line descriptions in the CLI, e.g.:
In the YANG file:
list ifs {
tailf:info "Configure interfaces";
...
}
list hosts {
tailf:info "Configure hosts";
...
}
list routing-protocol {
tailf:info "Routing protocols";
...

304

The CLI agent

}

In the CLI:
io(config)# config ?
Possible completions:
hosts
ifs
routing-protocol
io(config)# config

Configure hosts
Configure interfaces
Routing protocols

It is also possible to specify help texts for simpleTypes in the YANG files. For example:
typedef service-type {
type enumeration {
enum http { tailf:info "Web server"; }
enum smtp { tailf:info "Mail server"; }
}
}
typedef service-name {
tailf:info "Name of service";
type string;
}

When used in the CLI it looks like this:
io(config)# server ?

io(config)# server

It is also possible to modify the help texts for built-in types using the /clispec/$MODE/modifications/
typehelp element. For example:
integer

16.23. User defined wizards
If our configuration contains large structures that are nontrivial to configure in the CLI, we probably wish
to add tailor-made wizards to the CLI which aid the user in the process. Typically we want a wizard to
interact with the user, prompt the user, read some responses, and depending on the responses, populate
some structures with reasonable default values depending on the user's responses.
We can write our wizards either as executables or as callbacks. In this section we show how to add a user
to the AAA namespace using a shell script.
The wizard code needs to be able to perform the following tasks:
• It must be able to interact with the user, i.e. it must be able to read and write to the CLI terminal.
• It must be able to read and write to the db session which is used by the CLI. The CLI will start a db
session whenever it enters configuration mode, the wizard code must be able to read and write, not to
the database, but to the particular db session which is used by the running CLI. This is done in a shell
script using the maapi command which will attach to the current CLI session.
The command to invoke the wizard is added by editing the confd.cli file, and recompile it. The workings
of the confd.cli file is fully described in the UNIX man page clispec(5). A new wizard may be added by
adding the following in the confd.cli file:

305

The CLI agent

.....

.....

Configuration wizards
Configuration wizards

Create a Confd user

Create a Confd user and assign him/her to the proper group
to control what parts of the system he/she has access to.



./adduser.sh 






This way, once inside configuration mode, the command "wizard adduser" will be available. The type of
the callback is , indicating that the wizard is implemented as an executable.
For example, adduser.sh:
#!/bin/bash
## Ask for user name
while true; do
echo -n "Enter user name: "
read user
if [ ! -n "${user}" ]; then
echo "You failed to supply a user name."
elif maapi --exists "/aaa:aaa/authentication/users/user{${user}}"; then
echo "The user already exists."
else
break
fi
done
## Ask for password
while true; do
echo -n "Enter password: "
read -s pass1
echo
if [
echo -n
echo
else
echo -n
read -s
echo

"${pass1:0:1}" == "$" ]; then
"The password must not start with $. Please choose a "
"different password."
"Confirm password: "
pass2

if [ "${pass1}" != "${pass2}" ]; then
echo "Passwords do not match."

306

The CLI agent

else
break
fi
fi
done
groups=`maapi --keys "/aaa:aaa/authentication/groups/group"`
while true; do
echo "Choose a group for the user."
echo -n "Available groups are: "
for i in ${groups}; do echo -n "${i} "; done
echo
echo -n "Enter group for user: "
read group
if [ ! -n "${group}" ]; then
echo "You must enter a valid group."
else
for i in ${groups}; do
if [ "${i}" == "${group}" ]; then
# valid group found
break 2;
fi
done
echo "You entered an invalid group."
fi
echo
done
echo "Creating user"
maapi --create "/aaa:aaa/authentication/users/user{${user}}"
maapi --set "/aaa:aaa/authentication/users/user{${user}}/password" "${pass1}"
echo "Setting home directory to: /var/confd/homes/${user}"
maapi --set "/aaa:aaa/authentication/users/user{${user}}/homedir" \
"/var/confd/homes/${user}"
echo "Setting ssh key directory to: /var/confd/homes/${user}/ssh_keydir"
maapi --set "/aaa:aaa/authentication/users/user{${user}}/ssh_keydir" \
"/var/confd/homes/${user}/ssh_keydir"
maapi --set "/aaa:aaa/authentication/users/user{${user}}/uid" "1000"
maapi --set "/aaa:aaa/authentication/users/user{${user}}/gid" "100"
echo "Adding user to the ${group} group."
gusers=`maapi --get "/aaa:aaa/authentication/groups/group{${group}}/users"`
for i in ${gusers}; do
if [ "${i}" == "${user}" ]; then
echo "User already in group"
exit 0
fi
done
maapi --set "/aaa:aaa/authentication/groups/group{${group}}/users" \
"${gusers} ${user}"

307

The CLI agent

16.24. User defined wizards in C
We can write precisely the same wizard in C as well. This example makes use of the MAAPI interface,
described in the UNIX man page confd_lib_maapi(3) as well as in the user guide chapter "MAAPI Management Agent API". Thus to fully understand this section, the MAAPI documentation must be read.
Similar to the shell script wizard we must modify the clispec file "confd.cli" to indicate the name of a
program to execute. We have:

.....

Configuration wizards
Configuration wizards

Add a user
Add a user


/usr/local/bin/maapi_add_user

user




....



Using our modified "confd.cli", a command called "wizard cadduser" will be available in the CLI
configuration mode. When this CLI command is executed, the ConfD CLI will invoke the external program
called "/usr/local/bin/maapi_add_user".
This program will execute under a pseudo terminal (pty) which is connected to the actual CLI terminal.
Thus the external programs invoked by the CLI can manipulate the terminal - not just stdin and stdout.
The invoked program will have two environment variables set which can be used in the MAAPI interface
to create an attached MAAPI session towards the currently executing transaction in the CLI. Thus the
program must read "CONFD_MAAPI_USID" and "CONFD_MAAPI_THANDLE" from its environment.
Another thing which is different in the C implementation of a CLI wizard is that the C code must be
explicitly aware of which namespace it wants to manipulate. In our case, where we wish to add a user,
we wish to manipulate the AAA namespace, "http://tail-f.com/ns/aaa/1.1". The data model defining the
AAA namespace is included in a ConfD release and when the data model is compiled a .h file is generated
which contains the symbols defined in the namespace. This .h file must be included. Thus:
.....
#include "confd_lib.h"
#include "confd_maapi.h"
#include "aaa_bridge.h"
int main(int argc, char **argv)
{
int msock;
int debuglevel = CONFD_DEBUG;
struct in_addr in;

308

The CLI agent

struct sockaddr_in addr;
int usid, thandle;
char user[255], *pwd, home[255], sshdir[255];
char buf[BUFSIZ];
inet_aton("127.0.0.1", &in);
addr.sin_addr.s_addr = in.s_addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(4565);
confd_init("adduser", stderr, debuglevel);
confd_load_schemas((struct sockaddr*)&addr,
sizeof (struct sockaddr_in));
if ((msock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
confd_fatal("Failed to open socket\n");
if (maapi_connect(msock, (struct sockaddr*)&addr,
sizeof (struct sockaddr_in)) < 0)
confd_fatal("Failed to confd_connect() to confd: %s\n",
confd_lasterr());
usid = atoi(getenv("CONFD_MAAPI_USID"));
thandle = atoi(getenv("CONFD_MAAPI_THANDLE"));
if ((maapi_attach2(msock, aaa__ns, usid, thandle)) != CONFD_OK)
confd_fatal("Failed to attach: %s\n", confd_lasterr());
again0:
printf("Adding a user\n");
printf("Username: ");
if ((fgets(user,255, stdin)) == NULL)
exit(1);
user[strlen(user)-1] = 0;
if (maapi_cd(msock,thandle,"/aaa/authentication/users") != CONFD_OK)
confd_fatal("cannot CD: %s", confd_lasterr());
/* check if user exists */
if (maapi_exists(msock, thandle,"user{%s}", user) == 1) {
printf("user %s already exists: %s\n", user, confd_lasterr());
goto again0;
}
again:
if ((pwd = getpass("Password: ")) == NULL)
exit(1);
strcpy(buf, pwd);
if ((pwd = getpass("Confirm password: ")) == NULL)
exit(1);
if (strcmp(pwd, buf) != 0) {
printf("Password not confirmed\n");
goto again;
}
printf("Home: ");
if ((fgets(home,255,stdin)) == NULL)
exit(1);
home[strlen(home)-1] = 0;

printf("SSH dir: ");
if ((fgets(sshdir, 255, stdin)) == NULL)
exit(1);

309

The CLI agent

sshdir[strlen(sshdir)-1] = 0;

if (maapi_create(msock,thandle,"user{%s}",user) != CONFD_OK)
confd_fatal("failed to create user %s: %s\n", user, confd_lasterr());
if (maapi_set_elem2(msock,thandle,sshdir,"user{%s}/ssh_keydir",user) !=
CONFD_OK)
confd_fatal("failed to set ssh keydir: %s\n", confd_lasterr());
if (maapi_set_elem2(msock,thandle,pwd,"user{%s}/password",user) !=
CONFD_OK)
confd_fatal("failed to set password: %s\n", confd_lasterr());
if (maapi_set_elem2(msock,thandle,home,"user{%s}/homedir",user) !=
CONFD_OK)
confd_fatal("failed to set home: %s\n", confd_lasterr());
printf("user %s added successfully \n", user);
exit(0);
}

The code illustrates several points:
• It reads the above mentioned environment variables and calls maapi_attach2() to attach to the
transaction.
• The code includes the generated .h file from the AAA namespace. Furthermore, since the code is
manipulating the AAA namespace.
• It makes use of the libc function getpass() which opens "/dev/tty" to read a password without
echoing the characters entered by the user.
• It uses the MAAPI interface to read and write data. Once the program returns, the data written by the
program is still not committed. Not until the user executes the commit command in the CLI will the
data be actually written to the database opened by the CLI, whether running or the candidate.

16.25. User defined commands in C using the
C-API
A command can be implemented as a C-callback using the same API as used for actions, with some minor
modifications of the input parameters.
The arguments to the command are passed as string params where the first param is the full name of the
command, and the remaining params are the arguments that were used when invoking the command from
the CLI.
Maapi contains five CLI related functions which can be used for reading and writing to the CLI
from inside an action invoked from the CLI - maapi_cli_write, maapi_cli_printf, maapi_cli_prompt,
maapi_cli_prompt_oneof, and maapi_cli_read_eof.
Similar to commands implemented as executables, there is an option in the confd.cli file for specifying
that the command is implemented through a CAPI callback. It may look like this:

...

310

The CLI agent


C-API command example
C-API command example


testcommand



...


The capi tag tells confd that the command is implemented using CAPI and the cmdpoint is the name of
the action callback to invoke when the command is used from the CLI.

16.26. User defined commands as shell scripts
ConfD comes with a small C program called maapi. This program can be used inside shell scripts that
are defined as CLI commands, as exemplified in the add_user.sh script above. The maapi program is
thoroughly described in the man page maapi(1).
The program attaches itself to the current transaction using maapi_attach() (see confd_lib_maapi(3)) and
executes a single change.

16.27. Modifying built-in commands
There are a number of built-in commands which can be modified in a number of ways. There should not
be confused with the auto-rendered commands which cannot be modified in the same way. The autorendered commands are derived from the data-model at run-time and does not exist when confd is started.
The section below only relates to the built-in commands.

16.27.1. Renaming a command
A built-in command often consists of a command name part and a parameter part. For example, the
command config in C-mode has the name config and takes an optional parameter which can be either
terminal or exclusive.
It is possible to rename the command config but not the command argument. I.e. the following is possible,






The following will not work since the command is config not config terminal.






In the general case it is difficult to know what are commands and what are arguments to the command. The
above could have been defined as two commands config private and config exclusive without affecting
the CLI behavior. Using confd --cli-c-dump can be used for determining which part is a command name
and which parts are command arguments.

311

The CLI agent

16.27.2. Hiding the old, creating new
If you want to change the way a builtin command works, for example the config command above. There
is a trick that can be used. It consists of renaming the original command and hiding it. Then add your own
command and have it invoke the original hidden command.
Suppose we want to change the config command above to take the parameters private and exclusive instead
of terminal and exclusive. This is the way to do it:






Manipulate software configuration information
Manipulate software configuration information

/usr/local/bin/config.sh






private
non-locked editing of configuration


exclusive
locked editing of configuration







#!/bin/bash
case $1 in
private)
maapi --clicmd --no-hidden 'xxconfig terminal'
;;
shared)
maapi --clicmd --no-hidden 'xxconfig shared'
;;
*)
maapi --clicmd --no-hidden 'xxconfig terminal'
;;
esac

16.28. Tailoring show commands
There are several ways show commands can be tailored in the CLI. The data model can be modified,
explicit commands can be defined in the clispec files, the auto-rendered commands can be tweaked using

312

The CLI agent

clispec modifications, specific show callbacks can be defined in the clispec files, and show templates can
be defined in the clispec files.

16.28.1. How config="false" data is rendered
By default the CLI will create show commands in operational mode for displaying config="false" data.
Leaves and containers will be displayed as Name-Value pairs whereas list elements will be displayed as
tables, provided that the table will fit on the terminal screen.
A leaf element is rendered as follows:

Rendering of leaves and containers
Two different algorithms are used when rendering list elements. One for rendering a single element and
one for rendering multiple elements (either the entire table or a subset of the table).
When rendering a single element the following method is used:

313

The CLI agent

Rendering of a single list element
And when rendering a set of list elements the following is used:

314

The CLI agent

315

The CLI agent

Rendering of a set of list elements

16.28.2. Show templates
Show templates can be used in a few different ways. Either as a replacement for the default Name-Value
rendering, or for replacing the auto-rendered tables. By default only the Name-Value rendering is replaced
when a show template is defined, i.e. the show template will only be invoked for list elements if the table
is too wide for the screen (or if table view has been suppressed in some other way).
In order to also replace the default table rendering with the show template the auto-rendered table needs
to be suppressed. This is done using the YANG tailf:cli-suppress-table statement in the YANG file.
If a show template is used for rendering a table then the show template needs to render the table header,
this is done using a tailf:cli-show-template-legend statement. It also needs to suppress the default enter
text that is displayed for each instance. This is done by defining an empty tailf:cli-show-template-enter
template. Finally, a show template needs to be defined for the list element node that renders each table
line. A customized footer can be displayed by using a tailf:cli-show-template-footer statement.
The above will work fine when displaying an entire table. However, when displaying a single instance it
might be desirable to either display it as a table line, in which case a table header also needs to be displayed,
or in some other way. The problem is that the same show template is used in both situations.
The solution is to add a conditional in the template and display different texts in the two situations. For
example ( from the example/cli/show_template/jdemo.yang):
tailf:cli-show-template
"$(.legend_shown!=true?"
+"Address
Interface
"
+"$(.selected~=hwaddr?HW Address
)"
+"$(.selected~=permanent?Permant )"
+"$(.selected~=published?Published)"
+"$(.selected~=bignum?$(.show_bignum? Bignum ))\n"
+"============================"
+"$(.selected~=hwaddr?===================)"
+"$(.selected~=permanent?=========)"
+"$(.selected~=published?=========)"
+"$(.selected~=bignum?$(.show_bignum?========))\n)"
+"$(ip|ljust:15) $(ifname|ljust:9) - "
+"$(.selected~=hwaddr?$(hwaddr|ljust:17) )"
+"$(.selected~=permanent?$(permanent|ljust:8) )"
+"$(.selected~=published?$(published|ljust:7) )"
+"$(.selected~=bignum?$(.show_bignum?$(.humanreadable? "

In the example above the legend is only displayed if it has not already been displayed. This is achieved
by inspecting the built-in variable .legend_shown. The same behavior can be achieved by using the
substatement tailf:cli-auto-legend (see listing below). The example above also tests on the .selected
variable to determine if each column has been selected to be displayed using the select pipe command.
tailf:cli-show-template
"$(ip|ljust:15) $(ifname|ljust:9) - "
+"$(.selected~=hwaddr?$(hwaddr|ljust:17) )"
+"$(.selected~=permanent?$(permanent|ljust:8) )"
+"$(.selected~=published?$(published|ljust:7) )"
+"$(.selected~=bignum?$(.show_bignum?$(.humanreadable? " {
tailf:cli-auto-legend;
}

316

The CLI agent

It is possible to ask confd to perform a validation of the paths that appear in the templates in the YANG
files. This is done by running the command confd --cli-check-templates once confd has been started.

16.29. Change password at initial login
To force the user to change the password at initial login, or when the password has expired, a start script
can be used.
The start script is specified in the clispec file. Only one start script can be present, if more are present (for
example in different clispec files) then only one of them will be executed (unspecified which).
In the clispec file:



./startup.sh
$(user)




In startup.sh something along the lines of:
#!/bin/bash
user=$1
oldpass=`maapi --get "/aaa:aaa/authentication/users/user{${user}}/password"`
echo "oldpass=${oldpass}"
if [ ${oldpass} == '$1$Dd0v2$Rd899rbrbFTeHuEjAtzvW/' ]; then
echo "You need to set a new password"
## Ask for password
while true; do
echo -n "Enter password: "
read -s pass1
echo
if [ "${pass1:0:1}" == "$" ]; then
echo -n "The password must not start with $. Please choose a "
echo
"different password."
else
echo -n "Confirm password: "
read -s pass2
echo
if [ "${pass1}" != "${pass2}" ]; then
echo "Passwords do not match."
else
break
fi
fi
done
## create new transaction and write password

317

The CLI agent

confd_cmd -r -m -c \
"mset /aaa:aaa/authentication/users/user{${user}}/password ${pass1}"
fi
echo "Welcome!"

Note that the old password needs to be updated to your default startup password, or the test changed into
something a bit more sophisticated.

318

Chapter 17. The SNMP Agent
17.1. Introduction to the ConfD SNMP Agent
The ConfD integrated SNMP agent provides an environment where SNMP and other agents such as
NETCONF, Web UI, and CLI, coexist and use the same built-in database (CDB) for configuration storage
and the same set of instrumentation functions for controlling managed objects (MOs). Simple bindings
from SNMP MIB objects to YANG nodes is all that is needed to open up a configuration database to be
accessed by an SNMP manager.
The advantage of having an integrated SNMP agent in ConfD is that configuration data can be accessed
directly from the built-in database (CDB) or from user written managed objects without having to do the
tedious work of writing a separate set of instrumentation functions just for SNMP. One set of common
instrumentation functions is used for serving the NETCONF, CLI, Web UI, and SNMP agents.
confdc --mib2yang is used to make YANG models from MIBs. It also provides the necessary bindings
from MIB objects to nodes in the YANG model.
To go the opposite way, from YANG to MIBs, see Section 17.3, “Generating MIBs from YANG”.
The ConfD SNMP agent application provides the following features:
• An extensible SNMP agent that understands SNMPv1, SNMPv2c and SNMPv3.
• A MIB compiler that understands SMIv1 and SMIv2.
• Configuration data is specified in YANG models.
• Common instrumentation functions for controlling managed objects (MO's) via NETCONF, CLI, Web
UI, and SNMP agent.
• The SNMP agent uses ConfD AAA datarules to determine access to data, as well as using the standard
SNMP view based and user based access control mechanisms.
• The following MIBs are builtin in the ConfD SNMP agent:
• SNMPv2-MIB RFC 3418
• SNMP-FRAMEWORK-MIB RFC 3411
• SNMP-USER-BASED-SM-MIB RFC 3414
• SNMP-VIEW-BASED-ACM-MIB RFC 3415
• SNMP-COMMUNITY-MIB RFC 3584
• SNMP-TARGET-MIB and SNMP-NOTIFICATION-MIB RFC 3413
• SNMP-MPD-MIB RFC 3412
• TRANSPORT-ADDRESS-MIB RFC 3419
• SNMP-USM-AES-MIB RFC 3826
• IPV6-TC RFC 2465
• The following MIBs define the SMI language:

319

The SNMP Agent

• SNMPv2-SMI RFC 2578
• SNMPv2-TC RFC 2579
• SNMPv2-CONF RFC 2580
• RFC1155-SMI RFC 1155
• RFC-1212 RFC 1212
• RFC-1215 RFC 1215

17.1.1. Implementing MIBs
To set up an SNMP agent to manage objects in the MIB the following information must be provided:
• MIB
• YANG (.yang or .yin) file.
• Associations between objects in the MIB and nodes in the YANG module.
• Instrumentation functions (not needed for config data if CDB is used)
The MIBs are typically already existing standard or proprietary enterprise MIBs, but they can also be
generated from the YANG models.
The MIB compiler needs a mapping between the MIB object to nodes in the YANG module. This is done
by adding YANG statements to the data model, that associate YANG nodes with objects in the MIB. The
association statements can be written directly in the YANG module file, or in a separate YANG annotation
file (see tailf_yang_extensions(5)).
The confdc compiler verifies that the types of the SNMP objects and the types in the YANG module are
compatible.
There are three main use cases to consider when implementing a MIB with ConfD:
1. The MIB is given, and a YANG module is generated from the MIB.
The YANG modules and associations are generated with the confdc --mib2yang translator program.
The generated YANG modules will in this case resemble the structure of the MIBs.
2. The MIB and YANG module are both given (or written manually).
In this case, MIB associations should be written to bind MIB objects to the nodes in the YANG data
model. Statements tailf:snmp-name, tailf:snmp-oid, etc. are added either directly in the
YANG module or in a separate annotation file (see tailf_yang_extensions(5)).
3. The YANG module is given (or written manually), and the MIB is generated from it.
The MIBs are generated using the confdc --emit-mib command.

17.2. Agent Functional Description
The ConfD SNMP agent provides SNMP access to the data available through the ConfD management
backplane. The same data can also be accessed via other protocols/applications such as NETCONF, Web
UI, and CLI. This data can be stored in CDB, or made available by a data provider.

320

The SNMP Agent

SNMP has certain features that are not meaningful to model in YANG. There are also some requirements
on how the data is to be sorted in tables since SNMP operations require a strict lexicographical order of the
elements in a table. Below is a listing of some of the SNMP specific behaviors and what needs to be done:
• The RowStatus column in tables are handled by the SNMP agent and must not be part of the YANG
model. Rows with a RowStatus column set to 'notReady' or 'notInService' are temporarily stored in the
SNMP agent and will not show up in the database. Once activated they will be inserted into the database.
• SNMP requires objects that are stored in tables to be ordered in a strict lexicographical order. If we
have a list in a YANG module which is handled by an instrumentation function, the get_next()
callback function (which must be provided by the user), must return the elements in the same
lexicographical order as SNMP expects. If the order of the elements is not correct, then an SNMP
manager will not be able to correctly traverse the elements in a table. If the list statement has a
tailf:secondary-index with the name snmp, the agent will iterate over the table using this
index. Thus, the instrumentation code can reply with instances in SNMP lexicographical order when
the objects are retrieved over SNMP, and a more natural sort order when the objects are retrieved in
the CLI and other northbound interfaces.
• Tables with INDEX with dynamic length must have a length byte as part of the index. If the table index is
specified as IMPLIED, then the length byte is excluded from the index. The statement tailf:sortorder can be specified in lists and secondary indexes in the YANG module, to control whether index
elements should be ordered with the length byte included or not.
• Enumerations must have the same enumerated values in YANG and in the MIB. Note that the symbolic
string associated with the enum may be different in the YANG module and MIB.
• SNMP v3 has support for contexts. The ConfD SNMP agent uses "" as the default context where all
operations for this context are made towards the running database.
• Scalar variables of TestAndIncr are automatically handled by the agent.

17.2.1. Operation Overview
The following steps are needed to get a ConfD SNMP agent up and running:
1. Write a MIB module, generate one from a YANG module, or use an existing one.
2. Write a YANG module, generate one from a MIB module, or use an existing one.
3. Write associations that provide the mapping of MIB objects into YANG nodes.
4. Write instrumentation functions for nodes in the YANG module, or store data in CDB.
5. Compile the YANG module into an .fxs file.
6. Run the MIB together with the YANG .fxs file, and an optional mib annotations file (.miba, see
mib_annotations(5)), through the MIB compiler to produce a .bin file.
7. Configure the agent (confd.conf and initial MIB data). Specify which compiled MIBs are to be
loaded by the agent (.bin files) in confd.conf .
8. The produced .fxs file as well as the .fxs files for the built-in MIBs must be found in the loadPath
specified in confd.conf .
9. Start ConfD.

321

The SNMP Agent

17.2.2. MIBs and YANG
The basic objects in a MIB are scalar objects and table objects. Each MIB object must be mapped to a
node in a YANG module. Scalar MIB objects must be mapped to YANG leafs with matching types, so that
the agent can translate the value between the SNMP value and the internal value defined by the YANG
type. Tables in the MIB must be mapped to lists in YANG. The mapping between the MIB objects and the
YANG nodes is specified in the YANG module (or annotation file for the module) using tailf:snmpname and tail:snmp-oid statements. tailf:snmp-name specifies the symbolic name of a MIB
object, and tailf:snmp-oid specifies the corresponding OID.
Let us assume that we have the following MIB named SIMPLE-MIB containing a scalar and table:

-- a scalar
numberOfHosts OBJECT-TYPE
SYNTAX
INTEGER (0..65535)
MAX-ACCESS read-only
STATUS
current
DESCRIPTION
"Return the current number of hosts"
::= { simpleVariables 1 }
-- a table
hostTable OBJECT-TYPE
SYNTAX
SEQUENCE OF HostEntry
MAX-ACCESS
not-accessible
STATUS
current
DESCRIPTION
"The table of hosts."
::= { simpleTables 1 }
hostEntry OBJECT-TYPE
SYNTAX
HostEntry
MAX-ACCESS
not-accessible
STATUS
current
DESCRIPTION
"Information about a host."
INDEX
{ hostName }
::= { hostTable 1 }
HostEntry ::= SEQUENCE {
hostName
hostEnabled
hostNumberOfServers
hostRowStatus
}

DisplayString,
TruthValue,
Integer32,
RowStatus

hostName OBJECT-TYPE
SYNTAX
DisplayString
MAX-ACCESS not-accessible
STATUS
current
DESCRIPTION
"The unique index value of a row in this table."
::= { hostEntry 1 }
hostEnabled OBJECT-TYPE
SYNTAX
TruthValue
MAX-ACCESS
read-create

322

The SNMP Agent

STATUS
current
DESCRIPTION
"A bool value saying if host is enabled or not."
::= { hostEntry 2 }
hostNumberOfServers OBJECT-TYPE
SYNTAX
Integer32
MAX-ACCESS
read-only
STATUS
current
DESCRIPTION
"A read-only integer saying how many servers there currently are."
::= { hostEntry 3 }
hostRowStatus OBJECT-TYPE
SYNTAX
RowStatus
MAX-ACCESS
read-create
STATUS
current
DESCRIPTION
"The status of this conceptual row in the hostTable."
::= { hostEntry 4 }

An association must be written to bind the two SNMP objects (the scalar and the table) into YANG. Below
is an example of a simple YANG module with SNMP statements that maps to SNMP objects in the MIB.

Example 17.1. Simple YANG module
module simple {
namespace "http://tail-f.com/ns/simple";
prefix simple;
import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
typedef nameType {
type string {
length "min .. 255";
}
}
tailf:snmp-mib-module-name TAIL-F-TEST-MIB;
container simpleObjects {
leaf numberOfHosts {
type uint16;
mandatory true;
tailf:snmp-name numberOfHosts;
}
container hosts {
list host {
key name;
max-elements 64;
tailf:sort-order snmp;
tailf:snmp-name hostTable;

323

The SNMP Agent

tailf:snmp-row-status-column 4;
leaf name {
type nameType;
}
leaf enabled {
type boolean;
mandatory true;
tailf:snmp-name hostEnabled;
}
leaf numberOfServers {
type uint16;
mandatory true;
tailf:snmp-oid .3;
}
}
}
}
}

In the code listing above there is one variable numberOfHosts mapped to the SNMP scalar variable
numberOfHosts using the tailf:snmp-name statement. The numberOfServers object uses the
alternative notation with a tailf:snmp-oid statement. Which one to use is a matter of taste.
The list host is mapped to the SNMP table hostTable using tailf:snmp-name hostTable. It would
also be possible to specify the tailf:snmp-oid if preferred. Notice also that for tables which support
creation and deletion of rows through a RowStatus column, the statement tailf:snmp-row-statuscolumn can be given. (This is not necessary if the model will be used with an existing MIB, but it is
necessary for confdc --emit-mib to generate a writable table if the model is used for generating a new
MIB.) See Section 17.2.9, “The RowStatus column” for more details.
It is possible to map one YANG node to multiple SNMP objects. For example, if an SNMP table augments
another table, both these tables can be implemented in a single YANG list, where some leafs are mapped
to the base table, and some are mapped to the augmented table.
The following example illustrates the idea. The single YANG list 'interface' is mapped to the MIB tables
ifTable, ifXTable, and ipv4InterfaceTable:

list interface {
key index;
tailf:snmp-name 'ifTable'; // primary table
tailf:snmp-name 'ifXTable';
tailf:snmp-name 'IP-MIB:ipv4InterfaceTable';
leaf index {
type int32;
}
leaf description {
type string;
tailf:snmp-name 'ifDescr'; // mapped to primary table
}
leaf name {
type string;
tailf:snmp-name 'ifXTable:ifName';
}

324

The SNMP Agent

leaf ipv4-enable {
type boolean;
tailf:snmp-name
'IP-MIB:ipv4InterfaceTable:ipv4InterfaceEnableStatus';
}
...
}

17.2.3. Types
When the SNMP agent receives a request to GET an object, it will lookup the object in the compiled
MIB, and through the association information find the corresponding YANG node. Next, it will retrieve
the correct instances value from a data provider. This value will be encoded according to the type in the
YANG module. The SNMP agent translates this value to the corresponding SNMP value, and sends the
reply to the manager. For this translation to work, the types in the YANG module and MIB must match.
The following table shows how SMI data types are mapped to YANG datatypes. This mapping is used
internally in the agent in runtime, and also by the confdc --mib2yang program when a YANG module is
generated from a MIB. See the confd_types(3) man page for details about the XSD and confd types.

Table 17.1. SMI mapping to YANG types
SMI

YANG

C value type

XML example

OBJECT IDENTIFIER

yang:object-identifier

C_OID

1.3.6.1.2.1

IpAddress

inet:ipv4-address

C_IPV4

192.168.2.3

Unsigned32

uint32

C_UINT32

Gauge32

yang:gauge32

C_UINT32

Counter32

yang:counter32

C_UINT32

TimeTicks

yang:time-ticks

C_UINT32

Integer32

int32

C_INT32

Counter64

yang:counter64

C_UINT64

INTEGER { enums... }

enumeration

C_ENUMHASH

udp

INTEGER

int32

C_INT32

42

DisplayString

string

C_BUF/C_STR

Hello world!

OCTET STRING (with string
DISPLAY-HINT on the
form "Na" or "Nt")

C_BUF/C_STR

Hello world!

OCTET
STRING tailf:hex-list
(binary), Opaque

C_BINARY

4f:12:ff

IPV6-TC::Ipv6Address

inet:ipv6-address

C_IPV6

::213.180.94.158

SNMPv2TC::DateAndTime

yang:date-and-time

C_DATETIME

2006-08-17T16:30:53+02:00

SNMPv2TC::TruthValue

boolean,
enumeration C_BOOL,
(1), empty
C_ENUMHASH

SNMPv2TC::PhysAddress

yang:phys-address

C_BINARY

SNMPv2TC::MacAddress

yang:mac-address

C_BINARY

325

true

The SNMP Agent

SMI

YANG

C value type

SNMPv2TC::TimeStamp

yang:timestamp

C_UINT32

SNMPv2TC::TimeInterval

int32

C_INT32

SNMPv2-TC::TAddress tailf:octet-list

XML example

C_BINARY

(1) SNMPv2-TC::TruthValue is a bit special. At runtime, the agent can map it either to a normal
enumeration (which is how it is defined in SNMPv2-TC), to a boolean, to an empty leaf, or to a
presence-container. When confd --mib2yang is used to create the YANG module from a MIB, it uses the
enumeration mapping. This is also the recommended mapping. If a boolean is used, it cannot be part of
the INDEX in a table.
The following table shows how YANG data types are mapped to SMI datatypes. This mapping is used
internally in the agent in runtime, and also by the confdc --emit-mib program when a MIB is generated
from a YANG module.
If the association between the MIB and YANG module is written manually, the type mappings in this
table must be used.
Some of the more complex YANG types that cannot be easily translated to native SMI types are translated
into strings of the type "ConfdString" In this case, the human-readable string value is passed over SNMP.
These types cannot be used as INDEX in SNMP tables.
See the confd_types(3) man page for details about the XSD and confd types.

Table 17.2. YANG mapping to SMI types
YANG

SMI

C value type

Use as INDEX

int32

Integer32

C_INT32

yes

int16

Integer32
(-32768..32767)

C_INT16

yes

int8

Integer32 (-128..127)

C_INT8

yes

uint64

ConfdString

C_UINT64

yes

uint32

Unsigned32

C_UINT32

yes

uint16

Unsigned32 (0..65535)

C_UINT16

yes

uint8

Unsigned32 (0..255)

C_UINT8

yes

boolean

SNMPv2TC::TruthValue

C_BOOL

no

enumeration

INTEGER { enums... }

C_ENUMHASH

yes

string

OCTET STRING

C_BUF / C_STR

yes

decimal64

ConfdString

C_DECIMAL64

no

int64

ConfdString

C_INT64

no

union

ConfdString

depending on type

no

binary

OCTET STRING

C_BINARY

no

empty

SNMPV2TC::TruthValue

C_BOOL

no

identity

not supported

n/a

n/a

326

The SNMP Agent

YANG

SMI

C value type

Use as INDEX

yang:date-and-time

SNMPv2TC::DateAndTime

C_DATETIME

yes

yang:counter32

Counter32

C_UINT32

yes

yang:counter64

Counter64

C_UINT64

yes

yang:gauge32

Gauge32

C_UINT32

yes

yang:object-identifier

OBJECT IDENTIFIER

C_OID

yes

C_DOUBLE

no

xs:float,
xs:decimal

xs:double, ConfdString

inet:ipv4-address

IpAddress

C_IPV4

yes

inet:ipv6-address

IPV6-TC::Ipv6Address

C_IPV6

yes

inet:ip-address

OCTET STRING (SIZE C_IPV4 | C_IPV6
(4|16))

yes

inet:host

ConfdString

C_BUF / C_STR

no

inet:domain-name

ConfdString

C_BUF / C_STR

no

inet:port-number

Unsigned32 (0..65535)

C_UINT16

yes

inet:ipv4-prefix

OCTET STRING (SIZE C_IPV4PREFIX
(5))

yes

inet:ipv6-prefix

OCTET STRING (SIZE C_IPV6PREFIX
(17))

yes

inet:ip-prefix

OCTET STRING (SIZE C_IPV4PREFIX
(5|17))
C_IPV6PREFIX

| yes

tailf:size

OCTET STRING

C_BUF / C_STR

no

tailf:octet-list, tailf:hex- OCTET STRING
list

C_BINARY

yes

xs:date

ConfdString

C_DATE

no

xs:time

ConfdString

C_TIME

no

xs:duration

ConfdString

C_DURATION

no

xs:hexBinary

OCTET STRING

C_BINARY

no

xs:QName

not supported

n/a

n/a

A YANG presence container and a leaf of type empty can be mapped to a SMI scalar or columnar object
of type SNMPv2-TC::TruthValue. If the empty leaf or presence container exists, the SMI object is 'true',
and if it does not exist, but its parent exists, it has the value 'false'. Setting the SMI object to 'true' creates
the leaf or presence container, and setting it to 'false' deletes it.

17.2.4. Generating the YANG module
The confdc --mib2yang is used to generate a YANG (.yang) files from MIBs. The element structure in
the resulting YANG module will resemble the structure of the objects in the MIB.
If the MIB IMPORTs other MIBs, these MIBs must be available (as .mib files) to the compiler when
a YANG module is generated. By default, all MIBs in the current directory and all builtin MIBs (see
Section 17.1, “Introduction to the ConfD SNMP Agent”) are available. Since the compiler uses the tool
smidump to perform the conversion to YANG, the environment variable SMIPATH can be set to a colonseparated list of directories to search for MIB files.

327

The SNMP Agent

Example 17.2. Generating and compiling YANG from MIB
$ confdc --mib2yang -o SIMPLE-MIB.yang SIMPLE-MIB.mib
$ confdc -c -o SIMPLE-MIB.fxs SIMPLE-MIB.yang

Below is the generated YANG module. The structure of the YANG module resembles the structure of the
SIMPLE-MIB it was generated from.

Example 17.3. The YANG file generated by confdc --mib2yang
module SIMPLE-MIB {
namespace "http://tail-f.com/ns/mibs/SIMPLE-MIB/200702080000Z";
prefix SIMPLE-MIB;
tailf:id "";
tailf:snmp-mib-module-name SIMPLE-MIB;
import ietf-yang-types {
prefix yang;
}
import ietf-inet-types {
prefix inet;
}
import tailf-common {
prefix tailf;
}
import tailf-xsd-types {
prefix xs;
}
import SNMPv2-TC {
prefix SNMPv2-TC;
}
revision 2007-02-08 {
description "";
}
container SIMPLE-MIB {
container variables {
tailf:snmp-oid 1.3.6.1.4.1.24961.3.1.1;
leaf numberOfHosts {
type int32;
tailf:snmp-oid 1.3.6.1.4.1.24961.3.1.1.1;
}
}
container hostTable {
list hostEntry {
key hostName;
tailf:sort-order snmp;
tailf:snmp-oid 1.3.6.1.4.1.24961.3.1.2.1;
leaf hostName {
type hostNameType;
tailf:snmp-oid 1.3.6.1.4.1.24961.3.1.2.1.1.1;
}
leaf hostEnabled {
type SNMPv2-TC:TruthValue;
tailf:snmp-oid 1.3.6.1.4.1.24961.3.1.2.1.1.2;
}

328

The SNMP Agent

leaf hostNumberOfServers {
type int32;
tailf:snmp-oid 1.3.6.1.4.1.24961.3.1.2.1.1.3;
}
}
}
}
typedef hostNameType {
type string {
length "min .. 64";
}
}
}

17.2.5. Compiling the YANG modules
Compile the YANG modules representing MIBs the same way that any other YANG module is compiled,
using confdc .
Note that all YANG modules representing the builtin MIBs are available in $CONFD_DIR/src/confd/
snmp/yang directory. The parameter --yangpath can be given to the compiler to search this directory.

17.2.6. Compiling the MIBs

MIB compilation
The confdc compiler is used for compiling the MIB into a binary format that can be loaded by the ConfD
SNMP agent. The MIB is compiled with the YANG .fxs file with associations that map the YANG nodes
into objects in the MIB. The resulting file is a binary (.bin) file that is loaded into the ConfD SNMP agent.
$ confdc -c SIMPLE-MIB.mib simple.fxs

If the MIB IMPORTs other MIBs, these MIBs must be available (as compiled .bin files) to the compiler. By
default, all compiled MIBs in the current directory and all builtin MIBs are available. Use the parameters
--include-dir or --include-file to specify where the compiler can find the compiled MIBs.
Note that not every object in the MIB must have a mapping to a node in the YANG module. By using
a separate MIB annotation file, ConfD can be instructed how these missing objects should be treated by
the SNMP agent. The agent can treat the objects either as not implemented, or as implemented but nonexistent.
The format of an annotation line is object-name annotation, where object-name is the name of
an object type (column or scalar), and annotation has the form behavior=noSuchObject,
behavior=noSuchInstance, max_access=not_accessible, max_access=read_only.

329

The SNMP Agent

If a line is blank or starts with a # character, it is ignored. An object name may occur on several lines.
Example:
$ cat HOST-RESOURCES-MIB.miba
# tell the agent to not implement these objects
hrPartitionID
behavior=noSuchInstance
hrSWInstalledDate
behavior=noSuchObject
$ confdc -c HOST_RESOURCES-MIB.mib \
--mib-annotation HOST_RESOURCES.miba host-resources.fxs

17.2.7. Loading MIBs
The ConfD SNMP agent have the following built-in MIBs:
• SNMPv2-MIB, a mandatory MIB for an agent. This MIB is always loaded at start-up.
• SNMP-MPD-MIB, a mandatory MIB for an agent. This MIB is always loaded at start-up if the agent
is configured for SNMPv3.
• SNMP-FRAMEWORK-MIB, a mandatory MIB for an agent. This MIB is always loaded at start-up if
the agent is configured for SNMPv3.
• SNMP-COMMUNITY-MIB, handles SNMP v1 and v2c communities.
• SNMP-VIEW-BASED-ACM-MIB, handles the view based access control.
• SNMP-USER-BASED-SM-MIB, handles the user based access control.
• SNMP-TARGET-MIB, to be able to configure targets for SNMP traps.
• SNMP-NOTIFICATION-MIB, defines SNMP traps.
• IPV6-TC, defines some IPv6 specific TEXTUAL-CONVENTIONs.
• TRANSPORT-ADDRESS-MIB, defines some OBJECT-IDENTITYs for transport protocols. This MIB
module cannot be loaded as a built-in module in the agent. If some other MIB IMPORTs this MIB, then
it needs to be compiled and loaded as other non-built-in MIBs.
• SNMP-USM-AES-MIB, defines an OBJECT-IDENTITY for the AES privacy protocol. This MIB
module must not be loaded into the agent.
These MIBs of course must have their corresponding YANG .fxs files loaded in order for the SNMP agent
to work (see the next section). The MIBs themselves are not required to be loaded. The user decides which
MIBs should be loaded, and there may be reasons to not provide SNMP access to certain MIBs.
The MIBs SNMPv2-MIB, SNMP-MPD-MIB, and SNMP-FRAMEWORK-MIB, are always loaded into
the ConfD SNMP agent at start-up. These MIBs are required for an SNMP agent.
The other built-in MIBs are not loaded per default, which means that they cannot be accessed/configured
via SNMP but of course via for example CDB init files (see Section 5.8, “Loading initial data into CDB”)
NETCONF, or even CLI directly. If the intention is to have these MIBs loaded, they must be listed in
confd.conf without any explicit paths to where they are stored as shown in the example below.
Other MIBs (that are not built-in) are loaded by specifying their absolute or relative paths, or alternatively
the MIBs can be loaded from ConfDs loadPath. We recommend that these MIBs are loaded from the load
path. This is how other data model files (.fxs etc) are handled.

330

The SNMP Agent

The main reason for this recommendation is how MIBs can be dynamically reloaded. MIBs that are
explicitly listed are reloaded by giving the command confd --reload. If any MIB cannot be loaded for
whatever reason, ConfD halts. MIBs in the load path are reloaded using the data model upgrade functions,
see Chapter 13, In-service Data Model Upgrade.

Example 17.4. Specifying built-in MIBs to be loaded into the agent

true


SNMP-COMMUNITY-MIB.bin
SNMP-VIEW-BASED-ACM-MIB.bin
SNMP-USER-BASED-SM-MIB.bin
SNMP-TARGET-MIB.bin
SNMP-NOTIFICATION-MIB.bin

/etc/confd/mibs/SIMPLE-MIB.bin

true



17.2.8. Loading YANG modules for built-in MIBs
The SNMP agent has a few built-in MIBs that store its configuration data in CDB. The following .fxs
files defines namespaces for the built-in MIBs and must be loaded at start-up if the SNMP agent is enabled:
• SNMPv2-MIB.fxs, SNMPv2-SMI.fxs, SNMPv2-TC.fxs - contains base SNMPv2 types
• SNMP-FRAMEWORK-MIB.fxs
• SNMP-MPD-MIB.fxs
• SNMP-COMMUNITY-MIB.fxs
• SNMP-VIEW-BASED-ACM-MIB.fxs
• SNMP-USER-BASED-SM-MIB.fxs
• SNMP-TARGET-MIB.fxs
• SNMP-NOTIFICATION-MIB.fxs
Preferably a loadPath in the confd.conf file can be set to the directory where these files are installed.
If ConfD fails to load these files it will terminate with a fatal error.
The built-in .fxs files are delivered as pre-built, but the YANG source code is provided as well.
Tampering with these files is not advised and may result in a serious internal error. However we may wish
to recompile these YANG modules using the confdc compiler flag --export, to not expose the builtin MIB data to other ConfD internal protocols/applications such as NETCONF, CLI, and Web UI. The
YANG source code is provided for this reason.
Also, it is possible to provide external callpoints for the built-in MIB data to store the data in an
external database instead of CDB. If this is done and the SNMP Agent config is stored in an external

331

The SNMP Agent

database, ConfD must be told to pick up changes to the SNMP Agent config by means of the
maapi_snmpa_reload() function, see the ???? manual page. This is a drawback compared to storing
the data in CDB, since then, changes to the config will be automatically picked up by ConfD.

17.2.9. The RowStatus column
The rowstatus column for tables is always handled by the SNMP agent and should not be modeled in
the YANG modules. The tailf:snmp-row-status-column statement can be left out and the row
status column will be looked up by the compiler.
The RowStatus column in an SNMP table is used for reading the status of a conceptual row in a table and
for creating and deleting new conceptual rows in the table. The following values are always defined for
the row status:
• active (1) - indicates that the conceptual row is available.
• notInService (2) - indicates that the conceptual row is unavailable. This is a temporary state where the
row is stored in the SNMP agent and not in the database. A row with a RowStatus of 'notInService' can
be made 'active' which means that the row will be inserted into the database.
• notReady (3) - indicates that the conceptual row is missing information. This is a temporary state where
the row is stored in the SNMP agent and not in the database. A RowStatus of 'notReady' is returned to
indicate that the row is missing a value for one or more mandatory column(s). When the row have all
the mandatory values set, a RowStatus of 'notInService' will be returned instead of 'notReady'.
• createAndGo (4) - set by manager to create new row instance.
• createAndWait (5) - set by manager to create new row instance but not make it available. A RowStatus
of 'notInService' or 'notReady' is returned depending on if all values for the mandatory columns are
set or not.
• destroy (6) - set by manager to delete all instances in the conceptual row.
The createAndWait creates a new instance of a conceptual row in a temporary state where the row will
have a RowStatus set to 'notReady' or 'notInService' depending on if all the mandatory columns are set for
the column or not. It can be made 'active' and will then be inserted into the database. Note that there are no
guarantees that the row will exist more than 5 minutes (by default) in the temporary storage in the SNMP
Agent. The temporary cache used by the SNMP agent for this storage is volatile. The temporary storage
time can be configured in by setting temporaryStorageTime in confd.conf .
To delete a conceptual row the destroy value is used.

17.2.10. TestAndIncr
When a YANG module is generated from a MIB, and the MIB contains any scalar object of type
TestAndIncr, these objects are not translated into the YANG module, since they don't make sense outside
SNMP. Then, when the MIB is compiled, the compiler generates code so ConfD automatically handles
these objects. No coding is required.

17.2.11. MIB access and YANG config
Objects in MIBs can be read-only or writable. In YANG, nodes are marked as representing configuration
or non-configuration data.

332

The SNMP Agent

Read-only MIB objects
If a MIB object is read-only, it can be mapped to a configuration or non-configuration YANG node.
When a YANG module is generated from a MIB, all read-only MIB objects are translated into nonconfiguration YANG nodes.

Writable MIB objects
If a MIB object is writable, it is usually mapped to a configuration YANG node. However, in some cases,
the MIB object doesn't really represent configuration, but is rather writable operational data. An example
could be a scalar variable rebootRouter. When written to, the router will reboot. In order to support
this, non-configuration YANG nodes can be marked with tailf:writable true. Writable MIB
objects can be mapped into non-configuration YANG nodes that are marked with tailf:writable
true.
When a YANG module is generated from a MIB, writable MIB objects are translated into configuration
YANG nodes, unless the MIB object is marked as representing writable operational data in a MIB
annotation file (see mib_annotations(5)).
If the SNMP agent receives a SET PDU with one or more writable operational objects, it will start a readwrite transaction towards CONFD_OPERATIONAL. In this transaction, the agent will write all variables
from the PDU, and then it will commit the transaction. Instrumentation code needs to be written to handle
these writes.
See Section 7.8, “Writable operational data” for how these objects are implemented, and
examples.confd/snmpa/9-writable-operational in the ConfD examples collection for an
example of how this can be implemented.

17.2.12. Optional YANG nodes
There is no protocol support in SNMP to delete optional nodes. Conceptual table rows are typically created
and deleted by using a RowStatus column, but there is no standardized way to delete optional nodes.
One technique to handle this is to introduce a special value for the object, so that the object is deleted when
it is set to this special value. ConfD supports this technique with the YANG extension tailf:snmpdelete-value.
In order to use this technique, an optional leaf in the YANG model is mapped to a scalar or columnar object
in the MIB module. The data type definition of the MIB object allows the same values as the corresponding
YANG leaf, and in addition it also allows one extra value, not allowed by the YANG leaf. This special
value is also defined in the YANG model in the tailf:snmp-delete-value statement.
In the following example, we define a MIB object fooOptionalLeaf, and corresponding YANG leaf
foo-optional-leaf.

Example 17.5. SMI definition of an optional object
fooOptionalLeaf OBJECT-TYPE
SYNTAX
Integer32 (0..255)
MAX-ACCESS
read-create
STATUS
current
DESCRIPTION
"The special value zero means not used."
::= { fooEntry 3 }

333

The SNMP Agent

Example 17.6. YANG definition of an optional leaf
leaf foo-optional-leaf {
type int32 {
range "1..255";
}
tailf:snmp-delete-value 0;
tailf:snmp-name fooOptionalLeaf;
}

When the SNMP agent receives a SET request to set this object to '0', the leaf will be deleted.
If the tailf:snmp-delete-value statement has the substatement tailf:snmp-senddelete-value, the same special value will be returned on a GET request, instead of the default
noSuchInstance.

17.2.13. tailf:sort-order on tables
Tables in SNMP are strictly lexicographically ordered. An SNMP table is typically traversed with GETNEXT requests, where given a previous index of a row the next greater index is returned. Since the table
is specified in a YANG module and may be stored in an external database or perhaps as a managed object
(MO) written in C, it is important that the get_next() function returns the elements in correct order. If
the get-next function doesn't return the elements properly in order, SNMP will not work. If CDB is used to
store the data the ordering of the elements will be correct. Note that the tailf:sort-order statement
may have to be specified for indexes with dynamic length (see Section 17.2.8, “Loading YANG modules
for built-in MIBs”).
Types with dynamic length in SNMP like OCTET STRING will have a length indicator when they are
part of the INDEX, so the ordering for strings stored in such a table will be on length first, unless they are
declared as IMPLIED as in IMPLIED OCTET STRING. In this case the index will not have any length
indicator included, and the table should be sorted as normal.
The following values can be given to the tailf:sort-order statement:
normal

This is the default and means that the table is sorted using the key values. This value
should be used when the corresponding SNMP table does not have any INDEX object
with dynamic length.

snmp

This value means that when sorting, any key element of dynamic length will have the
length prepended to the value before sorting. It should be used if the corresponding
SNMP table has any INDEX object with dynamic length, and no IMPLIED object.

snmp-implied

This value is the same as snmp, except that the last key element will not prepend the
length indicator to the key value. It should be used if the corresponding SNMP table
has any IMPLIED INDEX.

Here's an example of a MIB table with a DisplayString with dynamic length as index.

Example 17.7. simple.mib
hostTable OBJECT-TYPE
SYNTAX
SEQUENCE OF HostEntry
MAX-ACCESS
not-accessible
STATUS
current
DESCRIPTION

334

The SNMP Agent

"The table of hosts."
::= { simpleTables 1 }
hostEntry OBJECT-TYPE
SYNTAX
HostEntry
MAX-ACCESS
not-accessible
STATUS
current
DESCRIPTION
"Information about a host."
INDEX
{ hostName }
::= { hostTable 1 }
HostEntry ::= SEQUENCE {
hostName
hostEnabled
hostNumberOfServers
hostRowStatus
}

DisplayString,
TruthValue,
INTEGER,
RowStatus

If the list corresponding to this SNMP table is stored in CDB, the definition in the YANG module must
specify tailf:sort-order snmp so that the table is sorted correctly (with length indicator included).

Example 17.8. simple.yang

list host {
key name;
tailf:sort-order snmp;
leaf name {
type nameType;
}
leaf enabled {
type boolean;
mandatory true;
}
leaf numberOfServers {
type uint16;
mandatory true;
}
}

When the sort order is set to "snmp" or "snmp-implied" on a list, it affects the displayed sort order in
all northbound interfaces. Thus the list of hosts above will be sorted according to SNMP lexicographical
ordering, even in e.g. the CLI. Sometimes this may be confusing to users. This problem can be solved by
adding a tailf:secondary-index to the list:

Example 17.9. simple.yang with secondary index
list host {
key name;
tailf:secondary-index snmp {
tailf:index-leafs "name";

335

The SNMP Agent

tailf:sort-order snmp;
}
leaf name {
type nameType;
}
leaf enabled {
type boolean;
mandatory true;
}
leaf numberOfServers {
type uint16;
mandatory true;
}
}

When the SNMP agent traverses a table, it checks if there is a secondary-index with the reserved name
"snmp" defined for the table. If there is such an index, the agent traverses the table using this index.
In the example above, the host table is sorted in normal order, which means that "arthur" appears
before "ford". But since there is a secondary-index called snmp, the SNMP agent will use this index when
traversing the table, so that 4."ford" appears before 5."arthur" over SNMP.
Note that the presence of a secondary-index in the YANG module is not enough; the instrumentation code
must be prepared to perform the actual sorting. See confd_lib_dp(3) for details.

17.2.14. Enumerations
Enumerations in SNMP have textual representation mapped to an integer. A simple example would be
the definition for TruthValue:

Example 17.10. TruthValue from the SNMPv2-TC
TruthValue ::= TEXTUAL-CONVENTION
STATUS
current
DESCRIPTION
"Represents a boolean value."
SYNTAX
INTEGER { true(1), false(2) }

The confdc --mib2yang tool produces the following type definition for TruthValue:

Example 17.11. A typedef for TruthValue
typedef TruthValue {
type enumeration {
enum true {
value 1;
}
enum false {
value 2;
}
}
}

336

The SNMP Agent

17.2.15. Notifications
Notifications are defined by the NOTIFICATION-TYPE macro in SMIv2. There are two types of
notifications in SNMP: trap and inform. When the managed object needs to send trap notifications the
following functions should be called (from MO's written in C).

Example 17.12. Functions for sending notification from C
int confd_register_snmp_notification(
struct confd_daemon_ctx *dx, int fd,
const char *notify_name, const char *ctx_name,
struct confd_notification_ctx **nctx);
int confd_notification_send_snmp(
struct confd_notification_ctx *nctx, const char *notification,
struct confd_snmp_varbind *varbinds, int num_vars);

The confd_register_snmp_notification() function is typically called once to register the
parameters common to a set of notifications, and then the individual notifications are sent by calling
confd_notification_send_snmp(). The daemon context pointer dx is obtained by calling
confd_init_daemon(), and the socket file descriptor fd is a worker socket connected to the
ConfD daemon via a call to confd_connect(). See confd_lib_dp(3) man page for details about
these functions. Note also that a control socket must be connected to the ConfD daemon before calling
confd_register_snmp_notification().
The notify_name parameter matches the NotifyName in the snmpNotifyTable in the SNMPNOTIFICATION-MIB. Trap will be sent to targets pointed out by NotifyName. If NotifyName is ""; the
normal procedure defined in SNMP-NOTIFICATION-MIB is used, i.e. the trap is sent to all managers.
Otherwise, the NotifyName is used to find an entry in the SnmpNotifyTable which defines how to send
the notification (as an Inform or a Trap), and to select targets from SnmpTargetAddrTable (using the Tag).
The ctx_name is the name of the context. The default context is "".
The notification string is the notification name. For example "coldStart" or "warmStart".
This symbolic name of a notification must be defined in a MIB that is loaded into the agent.
If the empty string is used as notification name, the notification to send is constructed from the varbinds
array alone, which must then contain a value for the snmpTrapOID variable.
The varbinds array contains variable bindings for parameters that should be sent along in the
notification. The include file confd_lib.h defines data structures for these:

Example 17.13. SNMP varbind structures from confd_maapi.h
enum confd_snmp_var_type {
CONFD_SNMP_VARIABLE = 1,
CONFD_SNMP_OID
= 2,
CONFD_SNMP_COL_ROW = 3
};
struct confd_snmp_oid {
int oid[128];
int len;
};

337

The SNMP Agent

struct confd_snmp_col_row {
char column[256];
struct confd_snmp_oid rowindex;
};
struct confd_snmp_varbind {
enum confd_snmp_var_type type;
union {
char name[256];
struct confd_snmp_oid oid;
struct confd_snmp_col_row cr;
} var;
confd_value_t val;
};

Each varbind is either:
a variable

a symbolic name of a scalar variable referred to in the notification
specification.

a column

a symbolic name of a column variable. Rowindex is the index of the
specified column.

an OBJECT IDENTIFIER

for the instance of an object, scalar variable or column variable.

If a value is given it will be used. If it is not given (i.e. set to C_NOEXISTS) then the agent will look up
the value with a GET operation.

Example 17.14. Notification registration
int setup(struct confd_daemon_ctx *dctx, int workersock,
struct confd_notification_ctx **nctx)
{
if (confd_register_snmp_notification(dctx, workersock,
"std_v1_trap", "", nctx)) != CONFD_OK)
return CONFD_ERR;
return confd_register_done(dctx);
}

Example 17.15. Sending a coldStart notification
int test1(struct confd_notification_ctx *nctx)
{
return confd_notification_send_snmp(nctx, "coldStart", NULL, 0);
}

Example 17.16. Sending a notification with a varbind
int test2(struct confd_notification_ctx *nctx)
{
struct confd_snmp_varbind vb;
vb.type = CONFD_SNMP_VARIABLE;

338

The SNMP Agent

strcpy(vb.var.name, "myVariable");
CONFD_SET_INT32(&vb.val, 32);
return confd_notification_send_snmp(nctx, "notif1", &vb, 1);
}

The inform type notifications are similar to the trap type except there is an acknowledgment sent back
from the manager that received the inform. Two callbacks needs to be registered to receive the result of
the inform, and there's a separate function for sending an inform.
int confd_register_notification_snmp_inform_cb(
struct confd_daemon_ctx *dx,
const struct confd_notification_snmp_inform_cbs *cb);
int confd_notification_send_snmp_inform(
struct confd_notification_ctx *nctx, const char *notification,
struct confd_snmp_varbind *varbinds, int num_vars,
const char *cb_id, int ref);

The struct confd_notification_snmp_inform_cbs is defined as:
struct confd_notification_snmp_inform_cbs {
char cb_id[MAX_CALLPOINT_LEN];
void (*targets)(struct confd_notification_ctx *nctx,
int ref, struct confd_snmp_target *targets,
int num_targets);
void (*result)(struct confd_notification_ctx *nctx,
int ref, struct confd_snmp_target *target,
int got_response);
void *cb_opaque;
/* private user data */
};

In order to debug the notification sending process in ConfD, the /confdConfig/logs/
developerLogLevel in confd.conf(5) can be set to "trace".

17.3. Generating MIBs from YANG
The previous sections have discussed the scenario where there is an existing set of MIB files, and then
confdc --mib2yang is used to convert these to YANG with the associations that the agent needs.
If instead no MIBs exist, but a number of YANG files (complied to .fxs files), these can be translated
to MIB files (in SMIv2 syntax), using the --emit-mib option of confdc.
The normal operation of the translator is to select those nodes that have an tailf:snmp-oid statement,
and ignore the others. If the option --generate-oids is given (described later in this section), all
elements are selected, unless explicitly marked with tailf:snmp-exclude-object.
The value of the tailf:snmp-oid statement can be either a one-component suffix, for example ".4", or
a full OID, such as "1.3.6.1.4.1.12345" or "private.1.12345". If it's a suffix, a full OID should be specified
for some ancestor element, in the YANG module header, or using the --oid option.
In order to be selected for translation to the MIB file, an element must also match the module name. The
module name can be given as an tailf:snmp-mib-module-name statement in the YANG module
header, which is then inherited by all nodes, or using the --module option. It can also be specified on a
node, which then overrides the value from the header or ancestor nodes.
Since tables can not reside inside tables in SMI, lists containing lists are handled by moving the inner
lists up to top level.

339

The SNMP Agent

Nodes inside containers in lists are given OIDs directly below the table entry, and names which are the
concatenation of the path down from the table level. The containers should not have an OID.
If a RowStatus column is desired for a table, use the statement tailf:snmp-row-status-column
on the corresponding list. The statement's value should be the column number (i.e., the last OID component,
with no leading period). The object will be called rowstatus.

17.3.1. Translating a .fxs file to a MIB
The form of the translation command is shown below (using fictitious parameters):
confdc --emit-mib FOO-MIB.mib --oid enterprises.24961 -- foo.fxs
The basename of the output file (here, FOO-MIB) by default becomes the name of the module (with all
letters made upper case). The option --module can be used to specify the module name.
Any other .fxs files we depend on have to be given with -f, as usual.
confdc --emit-mib FOO-MIB.mib --oid enterprises.24961 -f types.fxs -- foo.fxs
To build the .bin file to be loaded by the ConfD SNMP agent, do
confdc -c FOO-MIB.mib foo.fxs -f types.fxs
During translation, problems are reported as warnings or errors. When an error occurs, translation
continues so that a complete MIB file is still produced, but the exit status from confdc is 1, rather than 0.

17.3.2. --generate-oids
With the option --generate-oids, the translator selects all nodes, inventing OIDs for the nodes which
don't already have an tailf:snmp-oid statement. If a node has a tailf:snmp-exclude-object
statement, it is ignored. The --generate-oids is useful when the original YANG module cannot be
modified.
By default, the OID suffixes of child elements are numbered consecutively, starting with 1. This can
be overridden with a suffix tailf:snmp-oid on a node. The suffixes of the following elements will
continue from that point.
A RowStatus element is always generated.
Since the MIB compiler (i.e., confdc -c when given a MIB file) needs to know the association between
MIB objects and YANG nodes, and this association is not present in the YANG module (if it was, there
would be no need for generating OIDs), we use YANG annotation files.
A YANG annotation file is used to define the mapping between YANG nodes and MIB objects. The
YANG annotation file can be written by hand, or generated from the YANG module. Since it is important
in SNMP that OIDs once assigned do not change, it is recommended to generate an initial version of a
YANG annotation file, and then update it manually as the YANG module evolves. The process to do this is:
• Compile the YANG module: confdc -c foo.yang
• Generate an initial YANG annotation file: confdc --emit-mib FOO-MIB.mib --oid experimental.1 -generate-oids --generate-yang-annotation foo.fxs
The YANG annotation file will be called foo-ann.yang.

340

The SNMP Agent

Once the initial YANG annotation file is generated, it should be kept and updated as the original YANG
module is updated. The MIB can then be generated as needed:
• Compile the YANG module and annotation file: confdc -c -a foo-ann.yang foo.yang
• Generate the MIB: confdc --emit-mib FOO-MIB.mib foo.fxs
• Compile the MIB: confdc -c FOO-MIB.mib foo.fxs

17.3.3. Lexical structure
At the start of the generated MIB file, a header of comments gives some information on how the file was
produced, including the confdc invocation, the namespace of the .fxs file, and the current date and time.
(Any -- strings are converted to ++ because the former cannot occur in SMI comments.)
The only field in the module header which can be filled in with information from the .fxs file is the first
DESCRIPTION field, which is taken from the YANG module's description statement, if one exists.
The remaining fields have the following format, in order to facilitate automatic editing of the values:

LAST-UPDATED "@LAST-UPDATED"
ORGANIZATION "@ORGANIZATION"
CONTACT-INFO "@CONTACT-INFO"
REVISION "@REVISION"
DESCRIPTION "@REVISION-DESCRIPTION"

17.3.4. Names
The names of the objects are constructed by joining all the parts of the full tag path of the nodes, capitalizing
each part. An alternative is to not capitalize, and join the parts with "-" (with the option --join-names
hyphen see the section called “Emit SMIv2 MIB options”).
If the statement tailf:snmp-name is used on a node, its value is taken as the full name of the object.
For containers and tables, it also becomes the first part of its children's names.
The characters '.' and '_' can occur in YANG identifiers but not in SNMP identifiers; they are converted
to '-', unless the option --join-names force-capitalize is given. In this case, such identifiers
are capitalized (to lowerCamelCase).
If generated names clash with each other (for example both /x/a/b-c and /x/a-b/c yielding the name
x-a-b-c), an error is reported.

17.3.5. Types
YANG types are mapped according to the table in Table 17.2, “YANG mapping to SMI types”. .
The type restrictions that deal with lengths ranges are translated. The remaining restrictions (pattern,
fraction-digits) are silently ignored.
If inet:ipv6-address is used, Ipv6Address is imported from IPV6-TC. Otherwise, imports are only made
from SNMPv2-SMI, SNMPv2-CONF and SNMPv2-TC.
Leafs with types which are not handled are skipped in the translation, with a warning.

341

The SNMP Agent

Identifiers which have an invalid syntax (for example, start with a digit) are kept in the translation, but
a warning is given.
If a leaf with type yang:counter64 is used as an index or as writable, a warning is given.

17.3.6. Miscellaneous
STATUS is current by default for all objects. To cause STATUS to be deprecated or obsolete,
use the YANG statement status with the corresponding value on the YANG node.
MAX-ACCESS is read-only for operational nodes (having config false;).
Actions are silently ignored.
Before each generated OBJECT-TYPE and OBJECT IDENTIFIER, a comment containing the word
"tagpath" indicates which YANG node the object corresponds to.
DESCRIPTION is copied from the .fxs file, when available (if the description statement is
present). For containers, they are emitted as comments instead (the string "--" is replaced with "- -").
description for nodes that are not translated into any OID are lost. Double quote characters, which
can't occur in DESCRIPTION, are replaced with single quotes.
For a string with a min length, but no max length, the max length is assumed to be 65535.
tailf:sort-order snmp-implied; results in the IMPLIED keyword, if appropriate for the type.
If a type containing negative values is used as an index, or if a string with unlimited length is used as an
index, a warning is given.
If a list uses tailf:sort-order normal, the child nodes may not appear in SNMP order when
listed, and so some elements may get lost, or confuse the manager. A warning is given for such cases.
If tailf:sort-order snmp-implied is used for one list list, which contains another list, the
last index of the outer list (with implied length) can no longer have implied length in SNMP, so agent
communication will most likely fail.
DEFVAL clauses are emitted for string and integer types (including bit types), but not for others.

17.4. Configuring the SNMP Agent
Configuration data for the ConfD SNMP agent consists of:
• data in confd.conf
• data for built-in MIBs
The configuration data in confd.conf is typically only configured once and then never changed. (It is
possible to change however).
To store initial data for the built-in MIBs, CDB init files can be used. CDB init files are loaded the first
time the system is started and the database is initialized with this data. See Section 5.8, “Loading initial
data into CDB”. Typically these files will define access rules and users of the agent. Updating the MIB
data is done directly from the northbound agents such as NETCONF or CLI. The confdc compiler flag -export can be used to grant access for applications / protocols such as NETCONF and CLI to modify
the built-in SNMP data. For this reason the YANG source for the built-in MIBs are provided so that they
can be recompiled with the --export flag.

342

The SNMP Agent

The data for the SNMP agent built-in MIBs are by default stored in CDB, but it is also possible have this
data in an external database. In this case the user needs to add external callpoints to the YANG modules
and recompile them.

17.4.1. Configuration data in confd.conf
There are a few elements that must be configured in confd.conf in order for the SNMP agent to start.
First of all the ConfD SNMP agent must be enabled, and it must have an address and a port that it will
try to bind to at start-up. If if fails to bind to the port, ConfD will fail to start. It should also have a list of
MIBs that should be loaded into the agent. If it fails to load any of the MIBs, ConfD will fail to start.
Several options can be given to control the behavior of the SNMP agent:
enabled

Whether or not the agent should be started.

ip, port

The IP address and port that the agent will bind and listen for
incoming requests to.

mibs

The MIBs that the agent should load at start-up.

snmpVersions

The version of the SNMP protocol that the agent will understand
(the supported versions are v1, v2c, and v3).

snmpEngineID,
snmpEngineMaxMessageSize

The engine identifier and max message size that this agent can
handle.

sysDescr

Description of the entity. The description should include the full
name and version identification of the system. See SNMPv2-MIB
for more information.

sysObjectID

The vendor's authoritative identification of the network
management subsystem contained in the entity. See SNMPv2-MIB
for more information.

sysServices

A value which identifies the set of services that this entity primarily
offers. See SNMPv2-MIB for more information.

sysORTable

An optional table with SNMP agent capabilities. These entries will
populate the real sysORTable in the SNMPv2-MIB.

Below is an example of a confd.conf file:

Example 17.17. Example of a confd.conf

true
0.0.0.0
161

SIMPLE-MIB.bin


true
true
false



343

The SNMP Agent

80:00:61:81:05:01


Tail-f ConfD agent
1.3.6.1.4.1.24961



This will enable the SNMP agent, which will listen to requests incoming to locally at port 161. The MIB
file SIMPLE-MIB.bin is loaded in the agent, and the agent will understand SNMP versions v1 and v2c,
but not v3.

17.4.2. Changing configuration data in confd.conf in
run-time
The data stored in confd.conf can be changed by modifying the file and then issuing a confd --reload
command. This will trigger an already running ConfD daemon to check its configuration data and make
the necessary changes. Certain changes like the SNMP agents IP address will trigger an internal restart of
the SNMP agent (ConfD will still remain up), other changes like the MIBs that are loaded can be done
without restarting the SNMP agent. It's thus possible to update the MIBs in a running ConfD SNMP agent
without restarting the SNMP agent.

Example 17.18. Old confd.conf content

true
0.0.0.0
161

SIMPLE-MIB.bin



Example 17.19. Updated confd.conf content

true
0.0.0.0
161

SIMPLE-MIB.bin
SIMPLE-MIB2.bin



The example above will on a confd --reload command unload all the loaded MIBs that were specified
on the old configuration, and load the MIBs specified in the updated configuration. The MIB SIMPLEMIB.bin is unloaded and then loaded again, which can be useful during development to update to a
newer version of the MIB.

17.4.3. Built-in MIB data
There is a set of standard MIBs which are used to control and configure an SNMP agent. These MIBs are
implemented in this agent. The user can control which of these MIBs are actually loaded into the agent, and

344

The SNMP Agent

thus made visible to SNMP managers. For example, in a non-secure environment, it might be a good idea
to not make MIBs that define access control visible. Note, the data that the MIBs define is used internally
in the agent, even if the MIBs are not loaded.
Any SNMP agent must implement the system group and the snmp group, defined in SNMPv2-MIB. This
MIB will be loaded by default.
An SNMPv3 agent must implement the SNMP-FRAMEWORK-MIB and SNMP-MPD-MIB. These MIBs
are also loaded by default, if the agent is configured for SNMPv3.
There are five other standard MIBs, which also may be loaded into the agent. These MIBs are:
• SNMP-TARGET-MIB and SNMP-NOTIFICATION-MIB which defines managed objects for
configuration of management targets, i.e. receivers of notifications (traps and informs). These MIBs can
be used with any SNMP version.
• SNMP-VIEW-BASED-ACM-MIB, which defines managed objects for access control. This MIB can
be used with any SNMP version.
• SNMP-COMMUNITY-MIB, which defines managed objects for coexistence of SNMPv1 and
SNMPv2c with SNMPv3. This MIB is only useful if SNMPv1 or SNMPv2c is used, possibly in
combination with SNMPv3.
• SNMP-USER-BASED-SM-MIB, which defines managed objects for authentication and privacy. This
MIB is only useful with SNMPv3.
Initial config data for communities, access rules, users etc. is preferably stored in CDB init files. They are
read once when the system is started for the first time and put in the database. The files must be located in
the dbDir. Typically a system have the following CDB init files for the built-in MIBs (the file name can
be anything but must end with the suffix _init.xml):
community_init.xml

Data for SNMP-COMMUNITY-MIB. Defines the communities that have
access to the system.

vacm_init.xml

Data for SNMP-VIEW-BASED-ACM-MIB. Defines view based access.

usm_init.xml

Data for SNMP-USER-BASED-SM-MIB. Defines user based access used
with authentication and privacy. This is only used with SNMP v3.

target_init.xml

Data for SNMP-TARGET-MIB. Defines trap targets.

notify_init.xml

Data for SNMP-NOTIFICATION-MIB. Defines notifications.

Below is an example of an init file to define a community within the agent.

Example 17.20. Example community_init.xml



public
public
public
80:00:61:81:05:01


345

The SNMP Agent


permanent



SNMP-TARGET-MIB
The following values are supported for the object snmpTargetAddrTDomain:
• SNMPv2-TM::snmpUDPDomain UDP over IPv4
• TRANSPORT-ADDRESS-MIB::transportDomainUdpIpv4
snmpUDPDomain above)

UDP

over

IPv4

(same

as

• TRANSPORT-ADDRESS-MIB::transportDomainUdpIpv6 UDP over IPv6

SNMP-USER-BASED-SM-MIB
The following authentication algorithms are supported:
• SNMP-USER-BASED-SM-MIB::usmNoAuthProtocol No Authentication Protocol.
• SNMP-USER-BASED-SM-MIB::usmHMACMD5AuthProtocol The HMAC-MD5-96 Digest
Authentication Protocol.
• SNMP-USER-BASED-SM-MIB::usmHMACSHAAuthProtocol The HMAC-SHA-96 Digest
Authentication Protocol.
The following privacy algorithms are supported:
• SNMP-USER-BASED-SM-MIB::usmNoPrivProtocol No Privacy Protocol.
• SNMP-USER-BASED-SM-MIB::usmDESPrivProtocol The CBC-DES Symmetric Encryption
Protocol.
• SNMP-USM-AES-MIB::usmAesCfb128Protocol The CFB128-AES-128 Privacy Protocol.

17.5. How the SNMP Agent Interacts with
ConfD
17.5.1. ConfD Sessions and Transactions
All data access to ConfD is done through user sessions. Once a user session is established, it can open
read-only or read-write transactions towards a data store or towards operational data.
All requests start transactions towards the running data store.
SNMP over UDP does not have a concept of a user session. Each packet is (in theory) handled
independently. However, for performance reasons, the SNMP agent creates a user session and a transaction
when it receives the first packet. It then caches the session and transaction, and if it gets a new packet with
the same source IP address, same UDP port, and same securityName, it reuses the session and transaction.
If no packet has been received during a 10 second period, the session and transaction are closed.

346

The SNMP Agent

The cache has a limit of 32 sessions. If the cache is full when the agent needs to create a new session, an
old session can be closed for this purpose, even though it was in use less than 10 seconds ago.
The confd.conf parameters for session limits can be used to limit the number of concurrent SNMP
user sessions or configuration transactions, but note that higher values than 32 (the session cache limit
described above) will not have any effect.

17.5.2. USM and VACM and ConfD AAA
When the SNMP agent receives a SNMP request, it determines the securityName and SNMP context for
the request. If SNMPv1 or SNMPv2c is used, the snmpCommunityTable is consulted to determine the
securityName and SNMP context. If SNMPv3 is used, the SNMP context is explicitly given in the request,
and the securityName is determined from the usmUserTable.
When the SNMP agent starts a user session in ConfD, it uses the securityName as the username, the string
"snmp" as ConfD AAA context, and no groups. If the username is a member of any of ConfD's AAA
groups, it will be placed in these groups. Otherwise, if there is a defaultGroup configured in confd.conf
, the user will be placed in this group. Otherwise, the user does not belong to any group.
Note that the user is authenticated by the SNMP agent, and not by ConfD's AAA.
For each SNMP object the user tries to access, VACM is consulted to see if the user's securityName has
access, in the given context. If it has, the SNMP agent will try to access the corresponding YANG object.
ConfD's normal AAA authorization is consulted to see if the groups the user belongs to have access to
the YANG object.
Since both VACM and ConfD's AAA are consulted, a ConfD user can choose to use one of them, or
both. One usage strategy can be to add VACM rules which gives full access to everyone, and then rely
on ConfD's AAA datarules. Another strategy could be to have detailed rules in VACM, and then give full
access to the "snmp" context in ConfD's AAA.

17.5.3. ConfD High Availability
If ConfD is run in HA mode, the SNMP variables sysUpTime, snmpEngineTime, and
snmpEngineBoots are automatically replicated. This means that if a slave ConfD takes over as master,
these variables will keep their values.
It is essential that each ConfD instance in the cluster has the same snmpEngineID configured. This
value is defined in confd.conf , and it is the responsibility of the user to make sure it has the same
value on all nodes in the cluster. However, if ConfD's configuration is stored in CDB (see Section 28.4.2,
“Storing ConfD configuration parameters in CDB”), then since CDB is replicated, the snmpEngineID
will always be the same in the cluster.

17.6. Running the SNMP Agent as a NET-SNMP
subagent
The ConfD integrated SNMP agent can run as subagent to the NET-SNMP master agent. This is useful in
scenarios where you want to use NET-SNMP agents for monitoring the host, or reuse other NET-SNMP
subagents in your solution.
The easiest way to run the agent as sub-agent is to configure the proxy alternative in NET-SNMP
snmpd.conf. (See the snmpd.conf man page) Make sure that you have created an access view with the
correct OID root. You need to add a proxy command entry to the snmpd.conf file.

347

The SNMP Agent

proxy [-Cn CONTEXTNAME] [SNMPCMD_ARGS] HOST OID [REMOTEOID]

Values for the proxy configuration:
• SNMPCMD_ARGS : these are the authentication parameters you want to pass to the ConfD SNMP
agent. Note that the original auth parameters will not be used. (See snmpcmd man page). In the
simplest configuration you specify a community string, for example -c secret where secret will be used
as community string for all requests forwarded to ConfD.
• HOST : IPv4-address[:port] : the IP address and the port of the ConfD SNMP agent. Make sure that
ConfD uses a different port than the standard ports which you probably have configured for the NETSNMP master agent.
• OID : the SNMP OBJECT-IDENTIFIER of the root of the tree managed by ConfD SNMP agent.
After adding the values in the snmpd.conf file, restart the snmpd service.
The example below will forward all requests for Tail-f specific objects to the ConfD agent running on
localhost on port 5000 using the communit string secret.
proxy -c secret localhost:5000 1.3.1.6.4.24961

348

Chapter 18. Web UI Development
18.1. Introduction
Web UI development is thought to be in the hands of the customer's frontend developers - they will know
best the requirements and how to fulfill those requirements in terms of esthetics, functionality and tool
chain (frameworks, libraries, external data sources and services).
ConfD comes with a nortbound interface in the shape of a JSON-RPC API. This API is designed with Web
UI applications in mind, and it complies with the JSON-RPC 2.0 specification [http://www.jsonrpc.org/
specification], while using HTTP/S as the transport mechanism.
The JSON-RPC API contains a handful of methods with well defined input method and params, along
with the output result.
In addition, the API also implements a Comet model, as long polling, to allow the client to subscribe to
different server events and receive event notifications about those events in near real time.
You can call these from a browser via AJAX (e.g. XMLHTTPRequest, jQuery [https://jquery.com/]) or
from the command line (e.g. curl [https://github.com/bagder/curl], httpie [https://github.com/jkbr/httpie]):
// with jQuery
$.ajax({
type: 'post',
url: '/jsonrpc',
contentType: 'application/json',
data: JSON.stringify({
jsonrpc: '2.0',
id: 1,
method: 'login',
params: {
'user': 'joe',
'passwd': 'SWkkasE32'
}
}),
dataType: 'json'
})
.done(function(data) {
if (data.result)
alert(data.result);
else
alert(data.error.type);
});

or
# with curl
curl \
-X POST \
-H 'Content-Type: application/json' \
-d '{"jsonrpc": "2.0", "id": 1, \
"method": "login", \
"params": {"user": "joe", \

349

Web UI Development

"passwd": "SWkkasE32"}}' \
http://127.0.0.1:8008/jsonrpc
# with httpie
http POST http://127.0.0.1:8008/jsonrpc \
jsonrpc=2.0 id:=1 \
method=login \
params:='{"user": "joe", "passwd": "SWkkasE32"}'

18.2. Example of a common flow
You can read in the JSON-RPC API chapter about all the available methods and their signatures, but here
is a working example of how a common flow would look like:
• login
• create a new read transaction
• read a value
• create a new webui (read-write) transaction, in preparation for changing the value
• change a value
• commit (save) the changes
• meanwhile, subscribe to changes and receive a notification
In the release package, under ${CONFD_DIR}/var/confd/webui/example, you will find working
code to run the example below.
/*jshint devel:true*/
!!!
The following code is purely for example purposes.
The code has inline comments for a better understanding.
Your mileage might vary.
!!!

//
//
//
//
//

define([
'lodash',
'bluebird',
'./JsonRpc',
'./Comet'
], function(
_,
Promise,
JsonRpc,
Comet
) {
'use strict';
// CHANGE AT LEAST THESE
// IN ORDER TO MAKE THIS EXAMPLE WORK IN YOUR SETUP
var jsonrpcUrl = '/jsonrpc', // 'http://localhost:8008/jsonrpc';
path = '/dhcp:dhcp/max-lease-time',
value = Math.floor(Math.random() * 800) + 7200;

350

Web UI Development

var log,
jsonRpc,
comet,
funs = {},
ths = {
read: undefined,
webui: undefined
};
// UTILITY
log = function(msg) {
document.body.innerHTML =
'' +
msg +
'' +
document.body.innerHTML;
};
// SETUP
jsonRpc = new JsonRpc({
url: jsonrpcUrl,
onError: function(method, params, deferred, reply) {
var error = reply.error,
msg = [method,
params,
reply.id,
error.code,
error.type,
error.message
];
if (method === 'comet') {
return;
}
window.alert('JsonRpc error: ' + msg);
}
});
comet = new Comet({
jsonRpc: jsonRpc,
onError: function(reply) {
var error = reply.error,
msg = [reply.id, error.code, error.type, error.message];
window.alert('Comet error: ' + msg);
}
});
// CALLS FOR A COMMON SCENARIO
funs.login = function() {
log('Logging in as admin:admin...');
return jsonRpc.call('login', {
user: 'admin',
passwd: 'admin'
}).done(function() {
log('Logged in.');
});
};

351

Web UI Development

funs.getSystemSetting = function() {
log('Getting system settings...');
return jsonRpc.call('get_system_setting').done(function(result) {
log(JSON.stringify(result, null, 1));
});
};
funs.newReadTrans = function() {
log('Create a new read-only transaction...');
return jsonRpc.call('new_read_trans', {
db: 'running'
}).done(function(result) {
ths.read = result.th;
log('Read-only transaction with th (transaction handle) id: ' +
result.th + '.');
});
};
funs.newWebuiTrans = function() {
log('Create a new webui (read-write) transaction...');
return jsonRpc.call('new_webui_trans', {
conf_mode: 'private',
db: 'candidate'
}).done(function(result) {
ths.webui = result.th;
log('webui (read-write) transaction with th (transaction handle) id: ' +
result.th + '.');
});
};
funs.getValue = function(args /*{th, path}*/) {
log('Get value for ' + args.path + ' in ' + args.th + ' transaction...');
if (typeof args.th === 'string') {
args.th = ths[args.th];
}
return jsonRpc.call('get_value', {
th: args.th,
path: path
}).done(function(result) {
log(path + ' is now set to: ' + result.value + '.');
});
};
funs.setValue = function(args /*{th, path, value}*/) {
log('Set value for ' + args.path +
' to ' + args.value +
' in ' + args.th + ' transaction...');
if (typeof args.th === 'string') {
args.th = ths[args.th];
}
return jsonRpc.call('set_value', {
th: args.th,
path: path,
value: args.value
}).done(function(result) {
log(path + ' is now set to: ' + result.value + '.');
});
};

352

Web UI Development

funs.validate = function(args /*{th}*/) {
log('Validating changes in ' + args.th + ' transaction...');
if (typeof args.th === 'string') {
args.th = ths[args.th];
}
return jsonRpc.call('validate_commit', {
th: args.th
}).done(function() {
log('Validated.');
});
};
funs.commit = function(args /*{th}*/) {
log('Commiting changes in ' + args.th + ' transaction...');
if (typeof args.th === 'string') {
args.th = ths[args.th];
}
return jsonRpc.call('commit', {
th: args.th
}).done(function() {
log('Commited.');
});
};
funs.subscribeChanges = function(args /*{path, handle}*/) {
log('Subcribing to changes of ' + args.path +
' (with handle ' + args.handle + ')...');
return jsonRpc.call('subscribe_changes', {
comet_id: comet.id,
path: args.path,
handle: args.handle,
}).done(function(result) {
log('Subscribed with handle id ' + result.handle + '.');
});
};
// RUN
Promise.resolve([
funs.login,
funs.getSystemSetting,
funs.newReadTrans,
function() {
return funs.getValue({th: 'read', path: path});
},
function() {
var handle = comet.id + '-max-lease-time';
comet.on(handle, function(msg) {
log('>>> Notification >>>\n' +
JSON.stringify(msg, null, 2) +
'\n<<< Notification <<<');
});
return funs.subscribeChanges({th: 'read', path: path, handle: handle});
},
funs.newWebuiTrans,
function() {
return funs.setValue({th: 'webui', path: path, value: value.toString()});
},
function() {
return funs.getValue({th: 'webui', path: path});
},

353

Web UI Development

function() {
return funs.validate({th: 'webui'});
},
function() {
return funs.commit({th: 'webui'});
},
function() {
return new Promise(function(resolve) {
log('Waiting 2.5 seconds before one last call to get_value');
window.setTimeout(function() {
resolve();
}, 2500);
});
},
function() {
return funs.getValue({th: 'read', path: path});
},
]).each(function(fn){
return fn().then(function() {
log('--------------------------------------------------');
});
});
});

//
//
//
//

Local Variables:
mode: js
js-indent-level: 2
End:

18.3. Example of a JSON-RPC client
In the example above describing a common flow, a reference is made to using a JSON-RPC client to make
the RPC calls.
An example implementation of a JSON-RPC client, used in the example above:
/*jshint devel:true*/
!!!
The following code is purely for example purposes.
The code has inline comments for a better understanding.
Your mileage might vary.
!!!

//
//
//
//
//

define([
'jquery',
'lodash'
], function(
$,
_
) {
'use strict';
var JsonRpc;
JsonRpc = function(params) {

354

Web UI Development

$.extend(this, {
// API
// Call a JsonRpc method with params
call: undefined,
// API (OPTIONAL)
// Decide what to do when there is no active session
onNoSession: undefined,
// Decide what to do when the request errors
onError: undefined,
// Set an id to start using in requests
id: 0,
// Set another url for the JSON-RPC API
url: '/jsonrpc',

// INTERNAL
makeOnCallDone: undefined,
makeOnCallFail: undefined
}, params || {});
_.bindAll(this, [
'call',
'onNoSession',
'onError',
'makeOnCallDone',
'makeOnCallFail'
]);
};
JsonRpc.prototype = {
call: function(method, params, timeout) {
var deferred = $.Deferred();
// Easier to associate request/response logs
// when the id is unique to each request
this.id = this.id + 1;
$.ajax({
// HTTP method is always POST for the JSON-RPC API
type: 'POST',
// Let's show  rather than just "jsonrpc"
// in the browsers' Developer Tools - Network tab - Name column
url: this.url + '/' + method,
// Content-Type is mandatory
// and is always "application/json" for the JSON-RPC API
contentType: 'application/json',
// Optionally set a timeout for the request
timeout: timeout,
// Request payload
data: JSON.stringify({
jsonrpc: '2.0',
id: this.id,
method: method,
params: params
}),
dataType: 'json',

355

Web UI Development

// Just in case you are doing cross domain requests
// NOTE: make sure you are setting CORS headers similarly to
// -// Access-Control-Allow-Origin: http://server1.com, http://server2.com
// Access-Control-Allow-Credentials: true
// Access-Control-Allow-Headers: Origin, Content-Type, Accept
// Access-Control-Request-Method: POST
// -// if you want to allow JSON-RPC calls from server1.com and server2.com
crossDomain: true,
xhrFields: {
withCredentials: true
}
})
// When done, or on failure,
// call a function that has access to both
// the request and the response information
.done(this.makeOnCallDone(method, params, deferred))
.fail(this.makeOnCallFail(method, params, deferred));
return deferred.promise();
},
makeOnCallDone: function(method, params, deferred) {
var me = this;
return function(reply/*, status, xhr*/) {
if (reply.error) {
return me.onError(method, params, deferred, reply);
}
deferred.resolve(reply.result);
};
},
onNoSession: function() {
// It is common practice that when missing a session identifier
// or when the session crashes or it times out due to inactivity
// the user is taken back to the login page
_.defer(function() {
window.location.href = 'login.html';
});
},
onError: function(method, params, deferred, reply) {
if (reply.error.type === 'session.missing_sessionid' ||
reply.error.type === 'session.invalid_sessionid') {
this.onNoSession();
}
deferred.reject(reply.error);
},
makeOnCallFail: function(method, params, deferred) {
return function(xhr, status, errorMessage) {
var error;
error = $.extend(new Error(errorMessage), {
type: 'ajax.response.error',
detail: JSON.stringify({method: method, params: params})
});

356

Web UI Development

deferred.reject(error);
};
}
};
return JsonRpc;
});
//
//
//
//

Local Variables:
mode: js
js-indent-level: 2
End:

18.4. Example of a Comet client
In the example above describing a common flow, a reference is made to starting a Comet channel and
subscribing to changes on a specific path.
An example implementation of a Comet client, used in the example above:
/*jshint devel:true*/
!!!
The following code is purely for example purposes.
The code has inline comments for a better understanding.
Your mileage might vary.
!!!

//
//
//
//
//

define([
'jquery',
'lodash',
'./JsonRpc'
], function(
$,
_,
JsonRpc
) {
'use strict';
var Comet;
Comet = function(params) {
$.extend(this, {
// API
// Add a callback for a notification handle
on: undefined,
// Remove a specific callback or all callbacks for a notification handle
off: undefined,
// Stop all comet notifications
stop: undefined,
// API (OPTIONAL)
// Decide what to do when the comet errors
onError: undefined,
// Optionally set a different id for this comet channel

357

Web UI Development

id: 'main-1.' + String(Math.random()).substring(2),
// Optionally give an existing JsonRpc client
jsonRpc: new JsonRpc(),
// Optionally wait 1 second in between polling the comet channel
sleep: 1 * 1000,
// INTERNAL
handlers: [],
polling: false,
poll: undefined,
onPollDone: undefined,
onPollFail: undefined
}, params || {});
_.bindAll(this, [
'on',
'off',
'stop',
'onError',
'poll',
'onPollDone',
'onPollFail'
]);
};
Comet.prototype = {
on: function(handle, callback) {
if (!callback) {
throw new Error('Missing a callback for handle ' + handle);
}
// Add a handler made of handle id and a callback function
this.handlers.push({handle: handle, callback: callback});
// Start polling
_.defer(this.poll);
},
off: function(handle, callback) {
if (!handle) {
throw new Error('Missing a handle');
}
// Remove all handlers matching the handle,
// and optionally also the callback function
_.remove(this.handlers, {handle: handle, callback: callback});
//
//
//
if

If there are no more handlers matching the handle,
then unsubscribe from notifications, in order to releave
the server and the network from redundancy
(!_.find(this.handlers, {handle: handle}).length) {
this.jsonRpc.call('unsubscribe', {handle: handle});

}
},
stop: function() {
var me = this,
deferred = $.Deferred(),
deferreds = [];

358

Web UI Development

if (this.polling) {
// Unsubcribe from all known notifications, in order to releave
// the server and the network from redundancy
_.each(this.handlers, function(handler) {
deferreds.push(me.jsonRpc('unsubscribe', {
handle: handler.handle
}));
});
$.when.apply($, deferreds).done(function() {
deferred.resolve();
}).fail(function(err) {
deferred.reject(err);
}).always(function() {
me.polling = false;
me.handlers = [];
});
} else {
deferred.resolve();
}
return deferred.promise();
},
poll: function() {
var me = this;
if (this.polling) {
return;
}
this.polling = true;
this.jsonRpc.call('comet', {
comet_id: this.id
}).done(function(notifications) {
me.onPollDone(notifications);
}).fail(function(err) {
me.onPollFail(err);
}).always(function() {
me.polling = false;
});
},
onPollDone: function(notifications) {
var me = this;
// Polling has stopped meanwhile
if (!this.polling) {
return;
}
_.each(notifications, function(notification) {
var handle = notification.handle,
message = notification.message,
handlers = _.where(me.handlers, {handle: handle});
// If we received a notification that we cannot handle,
// then unsubcribe from it, in order to releave

359

Web UI Development

// the server and the network from redundancy
if (!handlers.length) {
return this.jsonRpc.call('unsubscribe', {handle: handle});
}
_.each(handlers, function(handler) {
_.defer(function() {handler.callback(message);});
});
});
_.defer(this.poll);
},
onPollFail: function(error) {
switch (error.type) {
case 'comet.duplicated_channel':
this.onError(error);
break;
default:
this.onError(error);
_.wait(this.poll, this.sleep);
}
},
onError: function(reply) {
var error = reply.error,
msg = [reply.id, error.code, error.type, error.message].join(' ');
console.error('Comet error: ' + msg);
}
};
return Comet;
});
//
//
//
//

Local Variables:
mode: js
js-indent-level: 2
End:

360

Chapter 19. The JSON-RPC API
19.1. JSON-RPC
19.1.1. Protocol overview
The JSON-RPC 2.0 Specification [http://www.jsonrpc.org/specification] contains all the details you need
in order to understand the protocol but here is the short version.
A request payload typically looks like this:
{"jsonrpc": "2.0",
"id": 1,
"method": "subtract",
"params": [42, 23]}

where the method and params properties are as defined in this manual page.
A response payload typically looks like this:
{"jsonrpc": "2.0",
"id": 1,
"result": 19}

or
{"jsonrpc": "2.0",
"id": 1,
"error":
{"code": -32601,
"type": "rpc.request.method.not_found",
"message": "Method not found"}}

The request id param is returned as-is in the response to make it easy to pair requests and responses.
The batch JSON-RPC standard is dependent on matching requests and responses by id, since the server
processes requests in any order it sees fit e.g.:
[{"jsonrpc": "2.0",
"id": 1,
"method": "subtract",
"params": [42, 23]}
,{"jsonrpc": "2.0",
"id": 2,
"method": "add",
"params": [42, 23]}]

with a possible response like (first result for "add", second result for "substract"):
[{"jsonrpc": "2.0",
"id": 2,
"result": 65}
,{"jsonrpc": "2.0",
"id": 1,
"result": 19}]

361

The JSON-RPC API

19.1.2. Common concepts
The URL for the JSON-RPC API is `/jsonrpc`. For logging and debugging purposes, you can add anything
as a subpath to the URL, for example turning the URL into `/jsonrpc/` which will allow you
to see the exact method in different browsers' *Developer Tools* - Network tab - Name column, rather
than just an opaque "jsonrpc".
For brevity, in the upcoming descriptions of each methods, only the input params and the output result
are mentioned, although they are part of a fully formed JSON-RPC payload.
Authorization is based on HTTP cookies. The response to a successful call to login would create a session,
and set a HTTP-only cookie, and even a HTTP-only secure cookie over HTTPS, named sessionid. All
subsequent calls are authorized by the presence and the validity of this cookie.
The th param is a transaction handle identifier as returned from a call to new_read_trans or
new_write_trans.
The comet_id param is a unique id (decided by the client) which must be given first in a call to the comet
method, and then to upcoming calls which trigger comet notifications.
The handle param needs to a semantic value (not just a counter) prefixed with the comet id (for
disambiguation), and overrides the handle that would have otherwise been returned by the call. This gives
more freedom to the client and set semantic handles.

Common errors
The JSON-RPC specification defines the following error code values:
• -32700 - Invalid JSON was received by the server. An error occurred on the server while parsing the
JSON text.
• -32600 - The JSON sent is not a valid Request object.
• -32601 - The method does not exist / is not available.
• -32602 - Invalid method parameter(s).
• -32603 - Internal JSON-RPC error.
• -32000 to -32099 - Reserved for application defined errors (see below)
To make server errors easier to read, along the numeric code, we use a type param that yields a literal
error token. For all application defined errors, the code is always -32000. It's best to ignore the code and
just use the type param.
{"jsonrpc": "2.0",
"id": 1,
"method": "login",
"params":
{"foo": "joe",
"bar": "SWkkasE32"}}

which results in:
{"jsonrpc": "2.0",
"id": 1,

362

The JSON-RPC API

"error":
{"code": -32602,
"type": "rpc.method.unexpected_params",
"message": "Unexpected params",
"data":
{"param": "foo"}}}

The message param is a free text string in English meant for human consumption, which is a one-to-one
match with the type param. To remove noise from the examples, this param is omitted from the following
descriptions.
An additional method-specific data param may be added to give further details on the error, most
predominantly a reason param which is also a free text string in English meant for human consumption. To
remove noise from the examples, this param is omitted from the following descriptions. But any additional
data params will be noted by each method description.

Application defined errors
All methods may return one of the following JSON RPC or application defined errors, in addition to others,
specific to each method.
{"type":
{"type":
{"type":
{"type":
{"type":

"rpc.request.parse_error"}
"rpc.request.invalid"}
"rpc.method.not_found"}
"rpc.method.invalid_params", "data": {"param": }}
"rpc.internal_error"}

{"type":
{"type":
{"type":
{"type":

"rpc.request.eof_parse_error"}
"rpc.request.multipart_broken"}
"rpc.request.too_big"}
"rpc.request.method_denied"}

{"type":
{"type":
{"type":
{"type":

"rpc.method.unexpected_params", "data": {"param": }}
"rpc.method.invalid_params_type", "data": {"param": }}
"rpc.method.missing_params", "data": {"param": }}
"rpc.method.unknown_params_value", "data": {"param": }}

{"type": "rpc.method.failed"}
{"type": "rpc.method.denied"}
{"type": "rpc.method.timeout"}
{"type": "session.missing_sessionid"}
{"type": "session.invalid_sessionid"}
{"type": "session.overload"}

19.1.3. FAQ
What are the security characteristics of the JSON-RPC api?
JSON-RPC runs on top the embedded web server (see "The web server" chapter), which accepts HTTP
and/or HTTPS.
The JSON-RPC session ties the client and the server via an HTTP cookie, named "sessionid" which
contains a randomly server-generated number. This cookie is not only secure (when the requests come

363

The JSON-RPC API

over HTTPS), meaning that HTTPS cookies do not leak over HTTP, but even more importantly this cookie
is also http-only, meaning that only the server and the browser (e.g. not the JavaScript code) have access
to the cookie. Furthermore, this cookie is a session cookie, meaning that a browser restart would delete
the cookie altogether.
The JSON-RPC session lives as long as the user does not request to logout, as long as the user is active
within a 30 minute (default value, which is configurable) time frame, as long as there are no severe server
crashes. When the session dies, the server will reply with the intention to delete any "sessionid" cookies
stored in the browser (to prevent any leaks).
When used in a browser, the JSON-RPC API does not accept cross-domain requests by default, but can
be configured to do so via the custom headers functionality in the embedded web server, or by adding a
reverse-proxy (see "The web server" chapter).

What is the proper way to use the JSON-RPC api in a cors setup?
The embedded server allows for custom headers to be se, in this case CORS headers, like:
Access-Control-Allow-Origin: http://webpage.com
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Origin, Content-Type, Accept
Access-Control-Request-Method: POST

A server hosted at http://server.com responding with these headers, would mean that the JSON-RPC API
can be contacted from a browser which is showing a web page from http://webpage.com, and will allow
the browser to make POST requests, with a limited amount of headers and with credentials (i.e. cookies).
This is not enough though, because the browser also needs to be told that your JavaScript code really wants
to make a CORS request. A jQuery example would show like this:
// with jQuery
$.ajax({
type: 'post',
url: 'http://server.com/jsonrpc',
contentType: 'application/json',
data: JSON.stringify({
jsonrpc: '2.0',
id: 1,
method: 'login',
params: {
'user': 'joe',
'passwd': 'SWkkasE32'
}
}),
dataType: 'json',
crossDomain: true,
// CORS specific
xhrFields: {
// CORS specific
withCredentials: true // CORS specific
}
// CORS specific
})

Without this setup, you will notice that the browser will not send the "sessionid" cookie on post-login
JSON-RPC calls.

What is a tag/keypath?
A tagpath is a path pointing to a specific position in a YANG module's schema.

364

The JSON-RPC API

A keypath is a path pointing to specific position in a YANG module's instance.
These kind of paths are used for several of the API methods (e.g. set_value, get_value, subscribe_changes),
and could be seen as XPath path specifications in abbreviated format.
Lets look at some examples using the following YANG module as input:
module devices {
namespace "http://acme.com/ns/devices";
prefix d;
container config {
leaf description { type string; }
list device {
key "interface";
leaf interface { type string; }
leaf date { type string; }
}
}
}

Valid tagpaths:
• `/d:config/description`
• `/d:config/device/interface`
Valid keypaths:
• `/d:config/device{eth0}/date` - the date leaf value within a device with an interface key set to eth0
Note how the prefix is prepended to the first tag in the path. This prefix is compulsory.

Restricting access to methods
The AAA infrastructure can be used to restrict access to library functions using command rules:

webui
webui
::jsonrpc:: get_schema
read exec
deny


Note how the command is prefixed with "::jsonrpc:: ". This tells the AAA engine to apply the command
rule to JSON-RPC API functions.
You can read more about command rules in "The AAA infrastructure" chapter in this User Guide.

What is session.overload error?
A series of limits are imposed on the load that one session can put on the system.
This reduces the risk that a session takes overs the whole system and brings it into a DoS situation.
The response will include details about the limit that triggered the error.
Known limits:

365

The JSON-RPC API

• only 10000 commands/subscriptions are allowed per session

19.2. Methods - commands
19.2.1. Method get_cmds
Get a list of the session's batch commands

Params
{}

Result
{"cmds": }
cmd =
{"params":
ConfD User Guide 6.6

confd_user_guide-6.6

Navigation menu

Versions of this User Manual:

Views

Navigation
