These days I’ve been thinking about how to improve the current implementation the data flows and connections systems of my library MICO (https://github.com/mico-corp/mico). Underneath, there is a small library I called FLOW, which is responsible of the data transit and callback calling of connected blocks in MICO. Just for those of you that don’t know about the library, it is a zero code tool focus in computer vision and robotics applications. For example, you can do something like this:
And… what does it relate to the title of this post!?…. Well, as I said I was trying to improve FLOW’s c++ interface to make it more intuitive, and also solve a some bugs that raised from current implementation. Let’s leave the introduction and start with the C++ code.
Variadic templates
What are variadic templates, well I do not want to get too technical. The official documentation is great https://en.cppreference.com/w/cpp/language/parameter_pack. As a summary, it is a template that accepts a list of template arguments, but all packaged in an argument, in a similar way than variadic argument does.
For example, you can create a recursive call consuming at each step one of the arguments.
#include <iostream>
// Last call of recursion.
template<typename T_>
void sayHeyVar(T_ _var) {
std::cout << _var << std::endl;
}
// variadic template function that accepts any amount of input arguments
template<typename T_, typename... Args_>
void sayHeyVar(T_ _var, Args_... _args) {
std::cout << _var << std::endl;
sayHeyVar(_args...);
}
// Simple class to illustrate a non basic type
struct Foo {
int internal_;
};
// Override operator << for Foo class
std::ostream& operator<<(std::ostream& _os, const Foo& _foo) {
_os << "I am foooooooo fighter number " << _foo.internal_;
return _os;
}
int main() {
sayHeyVar(1, 2.0f, "hola", 4.0, Foo{123});
}std::index_sequence
So… this a cool thing I have discovered recently, but it appeared officialy in C++14. It is basically a compile time list of integers. Particularly a 0 starting list of sequential integers. It is an specialization of std::integer_sequence and you can find more information here https://en.cppreference.com/w/cpp/utility/integer_sequence . We can extend previous code as adding the following function and you will see how the method prints sequentially integers from 0 to 4.
template<std::size_t ...Is>
void sayHeySequence(std::index_sequence<Is...>) {
sayHeyVar(Is...);
}
int main() {
sayHeySequence(std::make_index_sequence<5>{});
}but… Why would we want to use that… Well, I don’t know all the uses it has, but there is one usage that is extremelly cool, and it is related to variadic arguments. Let’s talk about this usage before moving to the toy code! Following code extends from previous one.
#include <iostream>
#include <vector>
template<typename T_>
void sayHeyVar(T_ _var) {
std::cout << _var << std::endl;
}
template<typename T_, typename... Args_>
void sayHeyVar(T_ _var, Args_... _args) {
std::cout << _var << std::endl;
sayHeyVar(_args...);
}
template<std::size_t ...Is>
void useVector(std::vector<int>& _data, std::index_sequence<Is...> const &) {
sayHeyVar(_data[Is]...);
}
int main() {
std::vector<int> data = { 3,1,2 };
useVector(data, std::make_index_sequence<3>{});
}Can’t you see it? useVector takes a runtime vector, and expands it as input arguments to be use in a variadic template method with variadic arguments! That is awesome! and expanded to me a new universe of possibilities.
The Toy code
Ok! Let’s start with the full code and we will cover it step by step! [DISCLAIMER, needs c++20!]
#include <iostream>
#include <functional>
#include <vector>
#include <map>
#include <utility>
#include <any>
using namespace std;
class A {
public:
template<class T_, typename... Arguments>
void registerCallback(std::vector<std::string> _tag, void(T_::* _cb)(Arguments... _args), T_* _obj) {
std::function<void(Arguments..._args)> fn = [_cb, _obj](Arguments..._args) {
(_obj->*_cb)(_args...);
};
registerCallback(_tag,fn);
}
template<typename T_>
T_ castData(std::any _data) {
return std::any_cast<T_>(_data);
}
template<typename... Arguments>
void registerCallback(const std::vector<std::string> &_tag, std::function<void(Arguments... _args)> _cb){
auto tmpCb = [&]<std::size_t ...Is> ( std::map<std::string, std::any> _data,
std::vector<std::string> _tags,
std::function<void(Arguments... _args)> _insideCb,
std::index_sequence<Is...> const &) {
std::vector<std::any> parsedData;
for (const auto& t : _tags) {
if (_data.find(t) != _data.end()) {
parsedData.push_back(_data[t]);
} else {
return; // One of the inputs is not present, do not proceed or will crash
}
}
_insideCb(castData<Arguments>(parsedData[Is])...);
};
realCb_ = std::bind(tmpCb, std::placeholders::_1, _tag, _cb, std::make_index_sequence<sizeof...(Arguments)>{});
}
void call(std::map<std::string, std::any> _data) {
realCb_(_data);
}
private:
std::function<void(std::map<std::string, std::any>)> realCb_;
};
class B {
public:
void superMethod(int _i1, float _i2, std::string _i3) {
std::cout << "I am even cooler " << _i1 << ", " << _i2 << ", " << _i3 << std::endl;
};
};
int main() {
A a;
// Test 1 ------------------------------------------------------------------------------------
std::function<void(int, std::string)> cb = [](int _i1, std::string _i2) {
std::cout << "WoW I am the callback! The result is " << _i1 << ", " << _i2 << std::endl;
};
a.registerCallback({ "i1", "i2" }, cb);
a.call({
{"i1", 1},
{"i2", std::string("hola")},
{"i2", std::string("adios")}
});
// Test 2 ------------------------------------------------------------------------------------
B b;
a.registerCallback({ "i1", "i2", "i3" }, &B::superMethod, &b);
a.call({
{"i1", 1},
{"i2", 2.0f},
{"i3", std::string("adios")}
});
return 0;
}
Ok, we are going to start “from Top to Bottom“. What we have at first intance is the main function. In there, we instantiate a class called A, in which we can register callbacks attached to a list of vectors that will be tags to name input data. After registering a callback, we can call the callback by using the “call” method and with a dictionary of any kind of data. We will see later, that for the first call we only use the entries of the dictionary existing in the list of tags when we attached the callback.
Later, we do the same thing, but in this case we use a method of a class and an instance of the class. Then we call it with a different dictionary and in this case we are using three inputs.
Ok, the behaviour is simple but… Have you realize that the callbacks had different inputs in size and types? That is awesome, because as we will see, the times are defined in compiletime but the trace of data is dynamic at runtime. And that is superb!
Let’s focus on the A class, we will ignore the first two methods at first instance. The important part occurs in the registerCallback method implementation:
template<typename... Arguments>
void registerCallback(const std::vector<std::string> &_tag, std::function<void(Arguments... _args)> _cb){
auto tmpCb = [&]<std::size_t ...Is> ( ..... ) {
......
};
realCb_ = std::bind(tmpCb, std::placeholders::_1, _tag, _cb, std::make_index_sequence<sizeof...(Arguments)>{});
}If we hide a bit of the code, what the method does at runtime is quite simple. It creates an object holding a lambda, and then binds the inputs of the method to it to store it for using it later. Just it, so where is the magic? Firstly, see that we are using one of the things we explained before, the std::index_sequence. In the previous example we picked a random number (5 actually), but in this case, we are using the “sizeof…(Arguments)” It means that the sequence will have the size of the number of arguments of the callback that we are registering. We will use that, to expand the input data when calling the callback outside (as we saw we do in the main function). Let’s see what happens inside of the lambda:
auto tmpCb = [&]<std::size_t ...Is> ( std::map<std::string, std::any> _data,
std::vector<std::string> _tags,
std::function<void(Arguments... _args)> _insideCb,
std::index_sequence<Is...> const &) {
std::vector<std::any> parsedData;
for (const auto& t : _tags) {
if (_data.find(t) != _data.end()) {
parsedData.push_back(_data[t]);
} else {
return; // One of the inputs is not present, do not proceed or will crash
}
}
_insideCb(castData<Arguments>(parsedData[Is])...);
};The code is super simple, and the biggest part is the central piece in which we pick the std::map of data and translate it into a simple std::vector. Because, as shown in previous vector, we can expand the vector using the std::index_sequence as easy as doing this fn(parsedData[Is]…)!!
Then the last line of code what is does is to call the callback that comes from the “registerCallback” method’s call and we will bind to the lambda. But… how do we make sure that the types are the correct ones? be cause we can apply a method to each of the arguments in the pack expansion. So in this single little piece of code (castData<Arguments>(parsedData[Is])...) There are happenings a lot of things. We are expanding parsedData to the N number of inputs arguments of the _insideCb callback which are packed in Arguments. And after doing the expansion, we are applying the method castData to each of the arguments, to adapt the type to the right one as defined in the Arguments packed list.
The result is a set of arguments with the right types that will be introduced as arguments in the desired callback. And this list of types and arguments adapts automatically to any callback we are registering. And even cooler! we are storing everything in a functor of defined type that does not depends on the declaration of the callback, so we can even store them in a vector or map to allow registering multiple callbacks!!
I am still amazed about this!
I did not explained some of the details, as for example, to make easier to separate the registering and the call of the callbacks I used std::any in the middle, that makes it much easier, and just need to call the castData<> method in the expansion of the arguments. Also, the other register method is just an overload to allow registering methods with objects, instead of just single functors.