Thursday, January 28, 2010

Alternatives to Plugins

I honestly haven't put much time into Boost.Extension and Boost.Reflection lately. My current work has no need for plugins of that type - and it's difficult to continue designing something without good current use cases to consider.

I've actually been surprised by the use cases many people propose to me in e-mail. Some people are looking for a full reimplementation of COM, and want plugins that can be compiled with a very different set of compiler options than the binary loading the plugin. In most cases, I felt like their use case did not justify such a complex solution.

So, do you really need C++ plugins? Consider the following:

  1. Do you need C++ performance? Or would a Boost.Python solution of integrating plugins be more effective? Python is certainly easier to deploy.
  2. Do you really need plugins? Or do you just need to compile your binary with different modules for different users?
  3. Are you shipping your source code to users to compile? If so, they can just compile in the plugins instead of putting them in a shared library.
If C++ plugins are still desired, here are my recommendations:
  1. Always compile the binary and plugins with the same compiler options.
  2. If you really need different compiler options:
    1. Consider building a distributed system, with the binary and the plugin communicating over network sockets or IPC.
    2. Consider using a plain C API, with very simple types being passed between the binary and the plugin (even structs can be a problem, since they may have different sizes depending on compiler options).
I plan to start posting here about some of my current work on the performance of optimization algorithms. I'm building a framework to automatically instrument and test different linear and nonlinear algorithms. I also hope to get back to dabbling in OpenGL...

7 comments:

Éric said...

Should we consider this post to be Boost.Extension's obituary?

Jeremy said...

I wasn't quite aiming for obituary - more of an explanation for the slow progress.

I'll write a post with more details.

iakov-minochkin said...

Good day!

I think it's useful to compile C++ modules on fly.

C++ already has tools for EDSL building, but such tools only for compile time. With such technology of plugins we can develop 'smart' application, wich can deside what kind of module required now or how to optimize existing one by full source code generation (using pre-designed domain-specific languages) and to load it by demand.

It means that C++ can becomes a language with support not only compile time metaprogramming, but runtime metaprogramming too.

Jeremy said...

I've been assisting a co-worker of mine who is using LLVM for this. Hopefully, we'll be able to publish some of the conclusions soon.

Garrett McGrath said...

Would you not consider this project to be officially dead?

For people looking for a bolt on plugin system would you consider poco's solution to be a viable alternative (http://pocoproject.org/docs/Poco.SharedLibrary.html)?

Garrett said...

Would you not consider this project to be officially dead?

For people looking for a bolt on plugin system would you consider poco's solution to be a viable alternative (http://pocoproject.org/docs/Poco.SharedLibrary.html)?

Jeremy said...

I haven't looked at Poco in about 4 years. I can't remember my analysis from that time. If you want to use it, you'll want to look carefully at how it handles allocation and deallocation across shared library boundaries (since this is different on different platforms), and be careful about the other items I mentioned here.

My overall opinion is the same as it was when I wrote this though. Most of the time when you think you need plugins using shared libraries, you'll have an easier time if you instead use Boost.Python or SWIG for non-performance critical code, or inter-process communication for code that needs to be fast. That's what I do now.

The initial project where I used these plugins should have used inter-process communication. My design was a mistake.

There are certainly times when plugins involving more than just a C interface (that is, passing full C++ objects between the libraries) are appropriate. C++ makes this very hard, and each of the available plugin libraries has significant gotchas or performance tradeoffs.