Wednesday, January 1, 2014

C++0x

I haven't worked on the Extension or Reflection libraries since 2008, primarily because of the difficulty of making it work in a predictable and consistent way across platforms. In general, the tricks involved with getting allocation and deallocation to work correctly across shared library boundaries required a lot of special case code on some platforms, and made it difficult to get the sort of simple API I was hoping for.

As such, I have made no effort since then to keep the library compiling and functioning with new compiler versions, nor with C++0x.

It would be possible to reimplement the Extension and Reflection code in a reasonably simply way with far less verbosity and templates using C++0x additions to the language. If it were not a requirement to make the code work across a variety of platforms and compilers, one could write a small custom version of the code for their own project.

In my current coding, I generally prefer to use either statically compiled binaries, or distinct processes communicating over RPC or IPC, rather than dynamically loading shared libraries. I have yet to find, in my own coding, any place where dynamic loading of shared libraries worked better than either of these two methods. The motivation for using dynamically loaded shared libraries was performance. I have never actually had a project where performance was critical, and I couldn't just compile static binaries.

This is not to say that there aren't use cases for dynamic loading of objects from shared libraries -- just that the complexities in building a cross-platform solution in C++ without a major runtime performance penalty were rather more than I was willing to put time into.

I've removed Boost.Extension from the Wikipedia article on plug-ins, so hopefully fewer of you will waste time attempting to get it to compile on newer compilers.

Wednesday, December 12, 2012

Hurricane Sandy Disaster Relief

I just open-sourced some code I put together for tracking disaster relief efforts for Hurricane Sandy at https://code.google.com/p/sandy-disaster-recovery. Since I put it together rather quickly, it's very much targeted at events with flooding and high winds. I'd like to find some time (or help) in generalizing the software more. It's currently being used by a few dozen organizations in the north-eastern United States to organize work at the homes of a few thousand people who have been affected by the hurricane.

I'm not a web developer, but I had some fun using App Engine for Python 2.7, Google Closure, and the Maps API to put this together. I may put together a post later about tricks I found for getting high performance out of App Engine applications.

Feel free to poke around in the code, fork it, or use it yourself -- it's available under the Apache 2.0 license.

Thursday, April 15, 2010

Questions

If you have questions on Boost.Extension, feel free to contact me:

Contact Jeremy Pack

Friday, March 26, 2010

Hanging Chain with Varying Weights

Here's another hanging chain scenario. The red links are a little longer and much heavier.

Thursday, March 25, 2010

The Hanging Chain Problem - or Catenary

Here's a video from some software I just wrote for the class I've been taking part time (you should never stop learning!)



Here's my description of the problem from my report:

Consider a chain hung from the ceiling. How can we predict its
shape as it hangs? Each link in the chain will naturally fall until it
is pulled back by neighboring links. As the links fall, the potential
energy of the chain decreases until the chain has reached it’s final
shape.
If we minimize the potential energy of the whole chain, while keep-
ing the links connected, we can find this final shape. The most obvious
method to solve this problem would be to slowly attempt to move the
links of the chain downward without unlinking them. Each link would
move a little bit at a time, eventually reaching its final position.
However, such a method could be very slow - especially for large
chains. First of all, the actual movement of each link is restricted
by the neighboring links, meaning that only infinitesimal movements
could be made at a time. In addition, moving any link will affect every
other link in the chain.


I'll add a few details of my solution once everyone else has turned their projects in.

Friday, January 29, 2010

Run-time Compilation - Performance

I put together an example of compiling code generated at run-time, and then loading it using Extension:

https://svn.boost.org/trac/boost/browser/sandbox/libs/extension/examples/runtime_compilation

I'm preparing some experiments to compare the performance of numerical code compiled in advance to that compiled at run-time. There will be three test cases involving complex matrix operations:

  1. The matrix sizes is known at compile-time, and can be used by the compiler for optimization. The matrix computations are done in the main binary.
  2. The matrix sizes are not known at compile-time. The matrix computations are done in the main binary using algorithms that work for arbitrarily-sized matrices.
  3. The matrix sizes are not known when the binary is compiled. When the binary loads a matrix, it will generate code on the fly to process the matrix. It will then compile the code into a shared library, load the shared library, and run the algorithm.
My hypothesis is that it will be possible to make the runtime-generated code faster than the arbitrarily-sized matrix code, for the following reasons:
  • The compiler will be able to make certain optimizations that it can only do when it knows the size of the arrays being looped over.
  • I'll be able to use fewer variables and more constants in the compiled code.
 I expect it to be slower than the specialized pre-compiled code for two reasons though:
  • Code in a shared library usually runs more slowly than code in the binary itself. One reason for this is that shared libraries do not know which address space they will be running at during run-time, and can't hardcode as many pointers and such as an executable. (Note that operating system libraries often have an optimization to avoid this problem, by using reserved address space)
  • Even if it did run as fast, it has to be compiled and loaded first.
Any opinions? I hope to post preliminary results in the next couple weeks or so.

Explanation

I consider Extension basically complete - quite a number of people are using it in projects. The main issue holding me back from submitting it for review by the Boost community is that I wanted to make the Reflection library part of Extension in a natural way, and I've been struggling at finding a balance between flexibility, performance and readability for the Reflection API. Also, of course, the Boost library review process is a bit painful - I'd need to block out at least 100 hours of my time to make sure the documentation, API, code, tests etc. were correct and easy to follow.

My previous post shouldn't be considered as much an obituary as an explanation of the slow progress lately. As I mentioned, I haven't been able to make much progress on the Reflection/Extension integration and new Extension features partially because it's difficult to design an API without having code that requires it.

If you find any features lacking in Extension, or aren't sure how to get the functionality you need, feel free to describe your use case to me - I can add the relevant functionality or documentation.

Also, always feel free to send patches. Extension is in the Boost sandbox:

https://svn.boost.org/trac/boost/browser/sandbox/libs/extension
https://svn.boost.org/trac/boost/browser/sandbox/boost/extension

Thursday, January 28, 2010

Alternatives to Plugins

I honestly haven't put much time into Boost.Extension and Boost.Reflection lately. My current work has no need for plugins of that type - and it's difficult to continue designing something without good current use cases to consider.

I've actually been surprised by the use cases many people propose to me in e-mail. Some people are looking for a full reimplementation of COM, and want plugins that can be compiled with a very different set of compiler options than the binary loading the plugin. In most cases, I felt like their use case did not justify such a complex solution.

So, do you really need C++ plugins? Consider the following:

  1. Do you need C++ performance? Or would a Boost.Python solution of integrating plugins be more effective? Python is certainly easier to deploy.
  2. Do you really need plugins? Or do you just need to compile your binary with different modules for different users?
  3. Are you shipping your source code to users to compile? If so, they can just compile in the plugins instead of putting them in a shared library.
If C++ plugins are still desired, here are my recommendations:
  1. Always compile the binary and plugins with the same compiler options.
  2. If you really need different compiler options:
    1. Consider building a distributed system, with the binary and the plugin communicating over network sockets or IPC.
    2. Consider using a plain C API, with very simple types being passed between the binary and the plugin (even structs can be a problem, since they may have different sizes depending on compiler options).
I plan to start posting here about some of my current work on the performance of optimization algorithms. I'm building a framework to automatically instrument and test different linear and nonlinear algorithms. I also hope to get back to dabbling in OpenGL...

Wednesday, June 24, 2009

Resource Management Across DLL Boundaries

Resource management across DLL boundaries in C++ is tricky. Here are the basic rules used in Boost.Extension:

  1. Anything allocated inside the DLL must be deallocated before the DLL is released. This means that if you pass a std::map into the DLL and modify it, you must delete (or at least clear()) the std::map before releasing the DLL.
  2. In order to use an interface across the DLL boundary, you basically have to guarantee that both the DLL and main executable were compiled with the same options - this is, for me, the primary weakness of C++ with DLLs.
  3. RTTI does not always function as expected across DLL boundaries. Check out the type_info classes to see how I deal with that.
  4. Always use virtual destructors for your interfaces - then you can safely delete them from the main binary or in the DLL.
It took a whole lot of trial and error to get this to work on different platforms.

Friday, August 1, 2008

Translation Bar

At least 50% of the visitors to this web site do not speak English as a first language. I have placed a Google Translation box on the left. Hopefully this will help make it easier to understand.

However, I realize that the translations it gives aren't very good - especially when it tries to translate source code. Don't misunderstand me - the Google software is quite impressive, but this is the type of text where a native speaker (who is also a programmer) is needed for a good translation.

If you would like to provide translations of any of the documentation content, I will post it here. The most important parts are the first two tutorials.