Skip to content

From 1.x to 2.x

Jump to bottom
Dario Izzo edited this page Jun 1, 2017 · 21 revisions

Motivations for a PaGMO 2.0

PaGMO 1.x development started in 2010 and the code basis quickly grew in the following years to a point where it soon became difficult to maintain. Besides, since it was created, a significant change in the c++ language happened: C++11. As a consequence it became suddenly clear that a complete rewrite of the code was necessary, giving us the chance to also rethink the whole API.

To summarize, PaGMO 1.x was:

  • Difficult to maintain
  • Difficult to further develop
  • Based on an old C++ style
  • Based on a not-up-to-our-standard software engineering infrastructure
  • Full of bugs

while PaGMO 2.0 is:

  • Coded in modern C++11 (with elements of C++14/C++17)
  • Easy to extend
  • Covered by unit tests to the largest possible extent
  • Using continuous integration CI in linux and windows from the very beginning (gcc, clang and MSVC compilers)
  • Offering a superior python experience when it comes to implementing UDAs and UDPs (user-defined problems and user-defined algorithms)
  • Fully integrated with NumPy on the Python side

In the following section we list the main differences in the overall design of the code base stemming from the above discussion.

Type Erasure

In PaGMO 2.x the polymorphic behavior of objects, e.g. a mathematical programming problem or an evolutionary algorithm, is achieved using the technique called type-erasure. The reason behind this choice is to free the user from the need to use / understand pointer semantics and clone methods. Let us see this as a concrete example from the user point of view. Lets create an std::vector of algorithms:

  • PaGMO 1.x:
using namespace pagmo;
std::vector<algorithm::base_ptr> algos;
algos.push_back(algorithm::base_ptr(new de(gen,0.9,0.9,3)));
  • PaGMO 2.x
using namespace pagmo;
std::vector<algorithm> algos;
algos.emplace_back(de{gen,0.9,0.9,3});

The new syntax (curly braces apart) makes use of the class pagmo::algorithm to erase the type of pagmo::de. In PaGMO 2.x any user developed algorithm (UDA) can be seen having the same type: algorithm. The same holds for other polymorphic classes: how cool is that :) !

This comes, of course, at a cost which, though, is not "paid" by the user: the code implementing polymorphism is somehow less intuitive as we are not using familiar concepts such as inheritance. The type-erasure technique we used is similar to what, in C++17, will be the std::any functionality, to which we added several features specific to our application.

See the great video "Better Code: Runtime Polymorphism - Sean Parent" explaining how and why to obtain the polymorphic behaviour of objects using type erasure.

type_erasure_poly

Some Template Meta-Programming

To allow the user to implement his own problem, algorithm etc.. PaGMO 2.x does not require anymore that his class inherits from some base class. The user can implement his own independent class / struct as long as he provides there a certain set of methods (or, in the meta-programming jargon, as long as the class / struct models a certain concept). To detect whether some class has or has not some method type_traits have been defined exploiting the sfinae rules as "commonly" done in template meta programming. The user does not have to care about this, though, he only needs to know how to implement, say an algorithm:

  • PaGMO 1.x:
class __PAGMO_VISIBLE null_algorithm:public base
{
    public:
        null_algorithm() {};
        base_ptr clone() const {return base_ptr(new null_algorithm(*this));};
        void evolve(population &pop) const {};
        std::string get_name() const {return "Null algorithm";}
    private:
        friend class boost::serialization::access;
        template <class Archive>
        void serialize(Archive &ar, const unsigned int)
        {
            ar & boost::serialization::base_object<base>(*this);
        }  
};
  • PaGMO 2.x:
struct null_algorithm
{
        null_algorithm() {};
        population evolve(population pop) const { return pop;};
        std::string get_name() const {return "Null algorithm";}
        template <typename Archive>
        void serialize(Archive &ar){}
};

A header only library

PaGMO is an open source project and makes use of quite some templates, as a consequence getting rid of the library complexity seemed like an obvious choice. Once again compile time is thus increased for the user, but we deemed this as an appropriate cost to pay.

Fewer Namespaces

In PaGMO 2.x the use of namespaces has been greatly reduced. Infact, apart from the main pagmo namespace, there are no additional ones the user should care about. After all, if the std library has only std::, we doubted that pagmo needed more. " C++ namespaces were not intended to be a design mechanism - they are there simply to prevent name clashes. You really don't want or need to use nested namespaces in 99.99% of situations. A good example of the correct use of namespaces in C++ is the C++ Standard Library. Everything in this quite large library is placed in a single namespace called std - there is no attempt or need to break the library up into (for example) an I/O sub-namespace, a math sub-namespace, a container sub-namespace etc."

Keep python and c++ API as similar as possible

While In PaGMO 1.x we developed in c++ without caring what the python API would then look like, PaGMO 2.x was developed as to keep c++ and python API as similar as possible. This means, for example, that when python makes copies so does c++ (except when performance critical): the use of const ref semantics in return value was thus reduced and several other c++ programming choice were made in this perspective.

Why should I move to PaGMO 2.0?

  • The old pagmo 1.0 will no longer be supported / maintained
  • PaGMO 2.0 installation process is straight forward (pip install pygmo or conda install pygmo in python or just use the headers, no library in c++)
  • PaGMO algorithms now all have a get_log() allowing to look what happens during evolves
  • pygmo 2.0 multiprocessing works in windows, linux and osx
  • slowly all features in PaGMO 1.0 will also be available in PaGMO 2.0 (help to implement/transfer them is always wanted)
  • You can implement problems and algorithms in python and be sure not to create nasty crashes with meta-problems / meta-algorithms
  • pagmo 2.0 uses a superior continuous integration chain ensuring compilation and unit tests pass in all platforms (64-32 bits) win linux osx
  • You can now implement gradients and hessians for your problem and have pagmo pass the information to the algorithm that care.
  • Thanks to cloudpickle you can define problems in one machine and send them around in a cluster.
  • Documentation is kept up to dat, is more complete and code snippets a guaranteed to be correct thanks to sphinx docutils extension.
  • PaGMO "speaks" NumPy: decision vectors, fitnesses, candidate solutions, populations, etc. are all represented or converted seamlessly to/from NumPy arrays, thus improving PaGMO's interoperability with the scientific Python ecosystem
Clone this wiki locally