Add support for instance-identifier
My goal was at least to make basic R/O operation work with ietf-alarms.
Then I didn't have an easy way of testing this code, so I made it work
for editing as well.
Limitations:
- You can set an instance-identifier which points to an input node of an
RPC/action, which sounds fishy, but that could be a libyang bug.
- Non-config lists works funnily, which might point to a bug somewhere
in our handling of data paths:
/> set example-schema:iid-valid /example-schema:users/userList
libyang[0]: Unexpected XPath token "]" ("]"). (path: Data location "/example-schema:iid-valid".)
libyang[0]: Invalid instance-identifier "/example-schema:users/userList[]" value - syntax error. (path: Data location "/example-schema:iid-valid".)
The following errors occured:
Message: Invalid instance-identifier "/example-schema:users/userList[]" value - syntax error.
XPath: Data location "/example-schema:iid-valid".
- XPath says that any key value must be quoted, but the CLI reuses the
existing leaf_data_ parser for list keys (even prior to this patch),
which means that a list indexed by, say, an enum looks like:
/example-schema/list[val=666]
...which is not a valid XPath. A valid XPath would look like this:
/example-schema/list[val='666']
This new feature hits the same problem; when the instance-identifier
gets parsed from the user's input and converted to a string, the usual
netconf-cli rules are followed, and the list keys are not quoted
unless they are strings. This means that these values cannot be stored
or committed because libyang correctly rejects unquoted key values in,
say, instance-identifier values.
Quite frankly I'm not even sure how come that this has not been a
problem during normal operation, but perhaps it's only because we've
always tested with string-based lists and nothing else. Sure, we have
partial unit tests for all sorts of data types, but nothing actually
checks the full stack from the parser's input all the way to a
backend.
Path completion was a challenge for me to get right. At first I wanted
to use a different, fresh ParserContext to make sure that we do not
affect the main (leaf) path's state at all. That did not work well even
when I tried to copy the "relevant" memebrs of ParserContext back to the
original parser. But it turns out that simply reusing the existing
ParserContext works perfectly well. The only caveat is the concept of
"cwd", where a leaf below a certain container would "expect" paths that
are relative to the original cwd (which means "not root"). That's just
broken. The easiest fix is to always enforce an absolute path. The
easiest way of getting there was a yet another "parser mode" for
absolute-paths-only.
Yay for the immediately-invoked-lambda, because I really wanted to use
an constexpr if in there.
Change-Id: I68302ad854eab43aa6c42dcb2693b71ca3d2bd48
Depends-on: https://gerrit.cesnet.cz/c/CzechLight/dependencies/+/6445
diff --git a/src/ast_values.cpp b/src/ast_values.cpp
index da65d7c..a7aa5fe 100644
--- a/src/ast_values.cpp
+++ b/src/ast_values.cpp
@@ -26,6 +26,21 @@
{
}
+instanceIdentifier_::instanceIdentifier_(const std::string& xpath)
+ : m_xpath(xpath)
+{
+}
+
+bool instanceIdentifier_::operator==(const instanceIdentifier_& other) const
+{
+ return this->m_xpath == other.m_xpath;
+}
+
+bool instanceIdentifier_::operator<(const instanceIdentifier_& other) const
+{
+ return this->m_xpath < other.m_xpath;
+}
+
binary_::binary_() = default;
binary_::binary_(const std::string& value)
diff --git a/src/ast_values.hpp b/src/ast_values.hpp
index 42608f7..208028b 100644
--- a/src/ast_values.hpp
+++ b/src/ast_values.hpp
@@ -54,6 +54,13 @@
std::string m_value;
};
+struct instanceIdentifier_ {
+ instanceIdentifier_(const std::string& xpath);
+ bool operator==(const instanceIdentifier_& b) const;
+ bool operator<(const instanceIdentifier_& b) const;
+ std::string m_xpath;
+};
+
enum class SpecialValue {
List,
LeafList,
@@ -74,6 +81,7 @@
empty_,
bits_,
identityRef_,
+ instanceIdentifier_,
special_,
double,
bool,
diff --git a/src/leaf_data.hpp b/src/leaf_data.hpp
index 0e0e499..0a7e465 100644
--- a/src/leaf_data.hpp
+++ b/src/leaf_data.hpp
@@ -39,6 +39,9 @@
-module >> node_identifier;
template <typename It, typename Ctx, typename RCtx, typename Attr>
+bool leaf_data_parse_data_path(It& first, It last, Ctx const& ctx, RCtx& rctx, Attr& attr);
+
+template <typename It, typename Ctx, typename RCtx, typename Attr>
struct impl_LeafData {
It& first;
It last;
@@ -95,6 +98,10 @@
{
return leaf_data_string.parse(first, last, ctx, rctx, attr);
}
+ bool operator()(const yang::InstanceIdentifier&) const
+ {
+ return leaf_data_parse_data_path(first, last, ctx, rctx, attr);
+ }
bool operator()(const yang::Empty) const
{
return x3::attr(empty_{}).parse(first, last, ctx, rctx, attr);
diff --git a/src/leaf_data_type.cpp b/src/leaf_data_type.cpp
index f97e382..aa36b8e 100644
--- a/src/leaf_data_type.cpp
+++ b/src/leaf_data_type.cpp
@@ -109,4 +109,11 @@
{
return this->m_allowedValues == other.m_allowedValues;
}
+InstanceIdentifier::InstanceIdentifier()
+{
+}
+bool InstanceIdentifier::operator==(const InstanceIdentifier&) const
+{
+ return true;
+}
}
diff --git a/src/leaf_data_type.hpp b/src/leaf_data_type.hpp
index c3c30c9..f1619ec 100644
--- a/src/leaf_data_type.hpp
+++ b/src/leaf_data_type.hpp
@@ -71,6 +71,10 @@
std::set<std::string> m_allowedValues;
};
struct LeafRef;
+struct InstanceIdentifier {
+ InstanceIdentifier();
+ bool operator==(const InstanceIdentifier& other) const;
+};
struct Union;
using LeafDataType = std::variant<
yang::String,
@@ -90,6 +94,7 @@
yang::IdentityRef,
yang::Bits,
yang::LeafRef,
+ yang::InstanceIdentifier,
yang::Union
>;
struct TypeInfo;
diff --git a/src/libyang_utils.cpp b/src/libyang_utils.cpp
index 09676d6..8ff4d65 100644
--- a/src/libyang_utils.cpp
+++ b/src/libyang_utils.cpp
@@ -40,9 +40,9 @@
return dec.number * std::pow(10, -dec.digits);
}
- leaf_data_ operator()(const libyang::InstanceIdentifier&) const
+ leaf_data_ operator()(const libyang::InstanceIdentifier& node) const
{
- throw std::runtime_error("instance-identifier is not supported");
+ return instanceIdentifier_{node.path};
}
template <typename Type>
diff --git a/src/path_parser.hpp b/src/path_parser.hpp
index 67345c7..5a4cf9e 100644
--- a/src/path_parser.hpp
+++ b/src/path_parser.hpp
@@ -256,7 +256,8 @@
enum class PathParserMode {
AnyPath,
DataPath,
- DataPathListEnd
+ DataPathListEnd,
+ DataPathAbsolute
};
template <>
@@ -274,6 +275,11 @@
using type = dataPath_;
};
+template <>
+struct ModeToAttribute<PathParserMode::DataPathAbsolute> {
+ using type = dataPath_;
+};
+
auto const trailingSlash = x3::rule<struct trailingSlash_class, x3::unused_type>{"trailingSlash"} =
x3::omit['/'];
@@ -300,10 +306,22 @@
initializePath.parse(begin, end, ctx, rctx, x3::unused);
dataPath_ attrData;
+ auto res = [](){
+ if constexpr (PARSER_MODE == PathParserMode::DataPathAbsolute) {
+ return absoluteStart;
+ } else {
+ return -absoluteStart;
+ }
+ }
// absoluteStart has to be separate from the dataPath parser,
// otherwise, if the "dataNode % '/'" parser fails, the begin iterator
// gets reverted to before the starting slash.
- auto res = (-absoluteStart).parse(begin, end, ctx, rctx, attrData.m_scope);
+ ().parse(begin, end, ctx, rctx, attrData.m_scope);
+ if (!res) {
+ x3::get<parser_context_tag>(ctx).m_suggestions.emplace(Completion{"/"});
+ return res;
+ }
+
auto dataPath = x3::attr(attrData.m_scope)
>> (dataNode<COMPLETION_MODE>{m_filterFunction} % '/' | pathEnd >> x3::attr(std::vector<dataNode_>{}))
>> -trailingSlash;
@@ -362,6 +380,7 @@
auto const anyPath = x3::rule<struct anyPath_class, AnyPath>{"anyPath"} = PathParser<PathParserMode::AnyPath, CompletionMode::Schema>{};
auto const dataPath = x3::rule<struct dataPath_class, dataPath_>{"dataPath"} = PathParser<PathParserMode::DataPath, CompletionMode::Data>{};
auto const dataPathListEnd = x3::rule<struct dataPath_class, dataPath_>{"dataPath"} = PathParser<PathParserMode::DataPathListEnd, CompletionMode::Data>{};
+auto const dataPathAbsolute = x3::rule<struct dataPath_class, dataPath_>{"dataPathAbsolute"} = PathParser<PathParserMode::DataPathAbsolute, CompletionMode::Data>{};
#if __clang__
#pragma GCC diagnostic push
@@ -465,7 +484,21 @@
auto const leafListElementPath = x3::rule<leafListElementPath_class, dataPath_>{"leafListElementPath"} =
dataPath;
-
+template <typename It, typename Ctx, typename RCtx, typename Attr>
+bool leaf_data_parse_data_path(It& first, It last, Ctx const& ctx, RCtx& rctx, Attr& attr)
+{
+ dataPath_ path;
+ auto res = dataPathAbsolute.parse(first, last, ctx, rctx, path);
+ if (res) {
+ // FIXME: list key escaping in XPath is different from netconf-cli.
+ // If the key(s) are not all strings, this will produce netconf-cli-style
+ // string like /foo:xxx[k1=blah][k2=666] instead of standard XPaths like
+ // /foo:xxx[k1='blah'][k2='666'], and these will cause troubles when passed
+ // to libyang later on.
+ attr = instanceIdentifier_{pathToDataString(path, Prefixes::WhenNeeded)};
+ }
+ return res;
+}
#if __clang__
#pragma GCC diagnostic pop
diff --git a/src/utils.cpp b/src/utils.cpp
index afb291e..fd2e297 100644
--- a/src/utils.cpp
+++ b/src/utils.cpp
@@ -127,6 +127,10 @@
{
return "an empty leaf";
}
+ std::string operator()(const yang::InstanceIdentifier&)
+ {
+ return "an instance identifier";
+ }
std::string operator()(const yang::Union& type)
{
std::ostringstream ss;
@@ -211,6 +215,11 @@
return ss.str();
}
+ std::string operator()(const instanceIdentifier_& data) const
+ {
+ return data.m_xpath;
+ }
+
template <typename T>
std::string operator()(const T& data) const
{
diff --git a/src/yang_schema.cpp b/src/yang_schema.cpp
index 8e282f1..2f046c7 100644
--- a/src/yang_schema.cpp
+++ b/src/yang_schema.cpp
@@ -246,6 +246,10 @@
resType.emplace<yang::Union>(std::move(resUnion));
break;
}
+ case libyang::LeafBaseType::InstanceIdentifier: {
+ resType.emplace<yang::InstanceIdentifier>();
+ break;
+ }
default:
using namespace std::string_literals;
throw UnsupportedYangTypeException("the type of "s +
diff --git a/tests/datastore_access.cpp b/tests/datastore_access.cpp
index e32076c..b7c4944 100644
--- a/tests/datastore_access.cpp
+++ b/tests/datastore_access.cpp
@@ -22,6 +22,7 @@
using OnInvalidRpcInput = sysrepo::ErrorWithCode;
using OnKeyNotFound = void;
using OnExec = void;
+using OnInvalidRequireInstance = DatastoreException;
#elif defined(netconf_BACKEND)
using OnInvalidSchemaPathCreate = std::runtime_error;
using OnInvalidSchemaPathDelete = std::runtime_error;
@@ -30,6 +31,7 @@
using OnInvalidRpcInput = std::runtime_error;
using OnKeyNotFound = std::runtime_error;
using OnExec = void;
+using OnInvalidRequireInstance = std::runtime_error;
#include "netconf_access.hpp"
#elif defined(yang_BACKEND)
#include <fstream>
@@ -42,6 +44,7 @@
using OnInvalidRpcInput = std::logic_error;
using OnKeyNotFound = DatastoreException;
using OnExec = std::logic_error;
+using OnInvalidRequireInstance = std::runtime_error;
#else
#error "Unknown backend"
#endif
@@ -532,6 +535,43 @@
REQUIRE(datastore.getItems("/example-schema:flags") == expected);
}
+ SECTION("instance-identifier")
+ {
+ SECTION("simple")
+ {
+ datastore.setLeaf("/example-schema:lol/up", true);
+ datastore.setLeaf("/example-schema:iid-valid", instanceIdentifier_{"/example-schema:lol/up"});
+ REQUIRE_CALL(mockRunning, write(sysrepo::ChangeOperation::Created, "/example-schema:lol/up", std::nullopt, "true"s, std::nullopt));
+ REQUIRE_CALL(mockRunning, write(sysrepo::ChangeOperation::Created, "/example-schema:iid-valid", std::nullopt, "/example-schema:lol/up"s, std::nullopt));
+ datastore.commitChanges();
+ DatastoreAccess::Tree expected{
+ {"/example-schema:iid-valid", instanceIdentifier_{"/example-schema:lol/up"}},
+ };
+ REQUIRE(datastore.getItems("/example-schema:iid-valid") == expected);
+ }
+
+ SECTION("to a non-existing list without require-instance")
+ {
+ datastore.setLeaf("/example-schema:iid-relaxed", instanceIdentifier_{"/example-schema:ports[name='A']/shutdown"});
+ REQUIRE_CALL(mockRunning, write(sysrepo::ChangeOperation::Created, "/example-schema:iid-relaxed", std::nullopt, "/example-schema:ports[name='A']/shutdown"s, std::nullopt));
+ datastore.commitChanges();
+ DatastoreAccess::Tree expected{
+ {"/example-schema:iid-relaxed", instanceIdentifier_{"/example-schema:ports[name='A']/shutdown"}},
+ };
+ REQUIRE(datastore.getItems("/example-schema:iid-relaxed") == expected);
+ }
+
+ SECTION("to a non-existing list with require-instance")
+ {
+ catching<OnInvalidRequireInstance>([&] {
+ datastore.setLeaf("/example-schema:iid-valid", instanceIdentifier_{"/example-schema:ports[name='A']/shutdown"});
+ datastore.commitChanges();
+ });
+ datastore.discardChanges();
+ }
+ }
+
+
#if not defined(yang_BACKEND)
SECTION("operational data")
{
diff --git a/tests/example-schema.yang b/tests/example-schema.yang
index 6e040ae..db27d57 100644
--- a/tests/example-schema.yang
+++ b/tests/example-schema.yang
@@ -341,4 +341,13 @@
}
}
+ leaf iid-valid {
+ type instance-identifier;
+ }
+
+ leaf iid-relaxed {
+ type instance-identifier {
+ require-instance false;
+ }
+ }
}
diff --git a/tests/leaf_editing.cpp b/tests/leaf_editing.cpp
index 5965110..302c8e4 100644
--- a/tests/leaf_editing.cpp
+++ b/tests/leaf_editing.cpp
@@ -72,6 +72,9 @@
schema->addLeaf("/", "mod:readonly", yang::Int32{}, yang::AccessType::ReadOnly);
schema->addLeaf("/", "mod:flags", yang::Bits{{"carry", "sign"}});
+ schema->addLeaf("/", "mod:iid", yang::InstanceIdentifier{});
+ schema->addLeaf("/mod:contA", "mod:x", yang::String{});
+ schema->addLeaf("/mod:contA", "mod:iid", yang::InstanceIdentifier{});
Parser parser(schema);
std::string input;
@@ -509,6 +512,47 @@
}
}
+ SECTION("instance-identifier") {
+ SECTION("toplevel")
+ {
+ input = "set mod:iid /mod:leafUint32";
+ expected.m_path.m_nodes.emplace_back(module_{"mod"}, leaf_{"iid"});
+ expected.m_data = instanceIdentifier_{"/mod:leafUint32"};
+ }
+
+ SECTION("deep to toplevel")
+ {
+ input = "set mod:contA/iid /mod:leafUint32";
+ expected.m_path.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ expected.m_path.m_nodes.emplace_back(leaf_{"iid"});
+ expected.m_data = instanceIdentifier_{"/mod:leafUint32"};
+ }
+
+ SECTION("deep to deep unprefixed")
+ {
+ input = "set mod:contA/iid /mod:contA/x";
+ expected.m_path.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ expected.m_path.m_nodes.emplace_back(leaf_{"iid"});
+ expected.m_data = instanceIdentifier_{"/mod:contA/x"};
+ }
+
+ SECTION("deep to deep mod-prefixed")
+ {
+ input = "set mod:contA/iid /mod:contA/mod:x";
+ expected.m_path.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ expected.m_path.m_nodes.emplace_back(leaf_{"iid"});
+ expected.m_data = instanceIdentifier_{"/mod:contA/mod:x"};
+ }
+
+ SECTION("absolute when nested")
+ {
+ cwd.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ input = "set iid /mod:contA/x";
+ expected.m_path.m_nodes.emplace_back(leaf_{"iid"});
+ expected.m_data = instanceIdentifier_{"/mod:contA/x"};
+ }
+ }
+
parser.changeNode(cwd);
command_ command = parser.parseCommand(input, errorStream);
REQUIRE(command.type() == typeid(set_));
@@ -683,6 +727,39 @@
input = "set mod:flags carry carry";
}
+ SECTION("instance-identifier non-existing node")
+ {
+ input = "set mod:iid /mod:404";
+ }
+
+ SECTION("instance-identifier non-existing node without leading slash")
+ {
+ input = "set mod:iid mod:404";
+ }
+
+ SECTION("instance-identifier node without leading slash")
+ {
+ input = "set mod:iid mod:leafUint32";
+ }
+
+ SECTION("instance-identifier to pseudo-relative unprefixed")
+ {
+ cwd.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ input = "set iid x";
+ }
+
+ SECTION("instance-identifier to pseudo-relative prefixed")
+ {
+ cwd.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ input = "set iid mod:x";
+ }
+
+ SECTION("instance-identifier without leading slash when nested")
+ {
+ cwd.m_nodes.emplace_back(module_{"mod"}, container_{"contA"});
+ input = "set iid mod:contA/x";
+ }
+
parser.changeNode(cwd);
REQUIRE_THROWS_AS(parser.parseCommand(input, errorStream), InvalidCommandException);
REQUIRE(errorStream.str().find(expectedError) != std::string::npos);
diff --git a/tests/path_completion.cpp b/tests/path_completion.cpp
index be676c9..8ad4b35 100644
--- a/tests/path_completion.cpp
+++ b/tests/path_completion.cpp
@@ -44,6 +44,7 @@
schema->addContainer("/", "example:system");
schema->addContainer("/example:system", "example:thing");
schema->addContainer("/", "example:system-state");
+ schema->addLeaf("/example:anoda", "example:iid", yang::InstanceIdentifier{});
auto mockDatastore = std::make_shared<MockDatastoreAccess>();
// The parser will use DataQuery for key value completion, but I'm not testing that here, so I don't return anything.
@@ -195,6 +196,42 @@
expectedCompletions = {"example:system-state/"};
expectedContextLength = 15;
}
+
+ SECTION("set example:anoda/iid ")
+ {
+ input = "set example:anoda/iid ";
+ expectedCompletions = {"/"};
+ expectedContextLength = 0;
+ }
+
+ SECTION("set example:anoda/iid /")
+ {
+ input = "set example:anoda/iid /";
+ expectedCompletions = {
+ "example:addresses",
+ "example:ano/",
+ "example:anoda/",
+ "example:bota/",
+ "example:leafInt ",
+ "example:list",
+ "example:ovoce",
+ "example:ovocezelenina",
+ "example:readonly ",
+ "example:system-state/",
+ "example:system/",
+ "example:twoKeyList",
+ "second:amelie/",
+ "second:fire/",
+ };
+ expectedContextLength = 0;
+ }
+
+ SECTION("set example:anoda/iid /example:read")
+ {
+ input = "set example:anoda/iid /example:read";
+ expectedCompletions = {"example:readonly "};
+ expectedContextLength = 12;
+ }
}
SECTION("get completion")
diff --git a/tests/utils.cpp b/tests/utils.cpp
index fa4ebed..05dfeb2 100644
--- a/tests/utils.cpp
+++ b/tests/utils.cpp
@@ -217,6 +217,16 @@
}
}
}
+
+ leaf iid-valid {
+ type instance-identifier;
+ }
+
+ leaf iid-relaxed {
+ type instance-identifier {
+ require-instance false;
+ }
+ }
}
)"s;
@@ -257,7 +267,9 @@
"name": "Aneta"
}
]
- }
+ },
+ "test-schema:iid-valid": "/test-schema:stuff[name='Xaver']/name",
+ "test-schema:iid-relaxed": "/test-schema:stuff[name='XXX']/name"
}
)"s;
@@ -392,6 +404,8 @@
{"/test-schema:users/userList[2]/name", std::string{"Aneta"}},
{"/test-schema:users/userList[3]", special_{SpecialValue::List}},
{"/test-schema:users/userList[3]/name", std::string{"Aneta"}},
+ {"/test-schema:iid-valid", instanceIdentifier_{"/test-schema:stuff[name='Xaver']/name"}},
+ {"/test-schema:iid-relaxed", instanceIdentifier_{"/test-schema:stuff[name='XXX']/name"}},
};
DatastoreAccess::Tree tree;
diff --git a/tests/yang.cpp b/tests/yang.cpp
index 0ba0af8..f56a27b 100644
--- a/tests/yang.cpp
+++ b/tests/yang.cpp
@@ -473,6 +473,16 @@
bit overflow;
}
}
+
+ leaf iid-valid {
+ type instance-identifier;
+ }
+
+ leaf iid-relaxed {
+ type instance-identifier {
+ require-instance false;
+ }
+ }
})";
TEST_CASE("yangschema")
@@ -793,6 +803,20 @@
type = yang::IdentityRef{{}};
}
+ SECTION("instance-identifier required")
+ {
+ node.first = "example-schema";
+ node.second = "iid-valid";
+ type = yang::InstanceIdentifier{};
+ }
+
+ SECTION("instance-identifier relaxed")
+ {
+ node.first = "example-schema";
+ node.second = "iid-relaxed";
+ type = yang::InstanceIdentifier{};
+ }
+
REQUIRE(ys.leafType(path, node) == yang::TypeInfo(type, std::nullopt, expectedDescription));
}
SECTION("availableNodes")
@@ -841,7 +865,10 @@
{"example-schema"s, "subLeaf"},
{"example-schema"s, "flagBits"},
{"example-schema"s, "leafFoodTypedef"},
- {"example-schema"s, "leafNoValidIdent"}};
+ {"example-schema"s, "leafNoValidIdent"},
+ {"example-schema"s, "iid-valid"},
+ {"example-schema"s, "iid-relaxed"},
+ };
}
SECTION("example-schema:a")
@@ -898,6 +925,8 @@
{"example-schema"s, "foodIdentLeaf"},
{"example-schema"s, "flagBits"},
{"example-schema"s, "interrupt"},
+ {"example-schema"s, "iid-relaxed"},
+ {"example-schema"s, "iid-valid"},
{"example-schema"s, "leafBool"},
{"example-schema"s, "leafDecimal"},
{"example-schema"s, "leafEnum"},
@@ -966,6 +995,8 @@
{boost::none, "/example-schema:ethernet/ip"},
{boost::none, "/example-schema:loopback"},
{boost::none, "/example-schema:loopback/ip"},
+ {boost::none, "/example-schema:iid-relaxed"},
+ {boost::none, "/example-schema:iid-valid"},
{boost::none, "/example-schema:interrupt"},
{boost::none, "/example-schema:leafBool"},
{boost::none, "/example-schema:leafDecimal"},