Elizafox's Ramblings

I was talking with my friend Anna about runtime modularity in Rust.

It turns out, unless you export a C API, or are willing to wade through mangled symbols, there's no real ability to do runtime modularity.

This is incredibly depressing and a sorry state of affairs.

It's also one that is, sadly, rather typical of modern languages.

The case for runtime modularity

Loading modules at runtime is incredibly useful. It lets us hotpatch functions, configure things and load functionality at runtime, and more.

IRC daemons do this as a standard practise since the dawn of time. Loading modules is integral to how a modern IRC daemon works.

Apache uses it to load modules at runtime. That way, you don't have to bloat your server with modules you'll never use, but they're always there if you need them, without recompiling. It's nifty!

So if something like this is so fundamental, why isn't it more widely supported?

The scourge of C

Because of name mangling, it's a complete pain to actually load things at runtime that aren't C symbols. C symbols aren't really mangled, per se (but see this and this), so loading them at runtime is fairly simple. As the article on name mangling says, we need mangling to actually express modern concepts such as monomorphisation and function overloading and such.

Unfortunately, pretty much no language makes it easy to mangle a symbol, especially at runtime. This is needed to use dlsym() or GetProcAddress() to load the symbol so we can use it.

Rust can't do it. C++ can't do it. D (to my knowledge) can't do it.

It's infuriating because the compiler knows how to do it. Yet it won't tell you how!!! Why, is beyond me.

The reason we have to do this name mangling crap to begin with is because of C. C hangs over all our heads as the lingua franca of all our systems. We must bend ourselves to its evil whims to accomplish runtime loading, because there is literally no other way.

Which means that we're all forced to speak C, or use horrible unportable hacks like reading the entire library's symbol table and demangling it.

Boost.DLL is one such package that does such cursed things. It can do a mangled import from a dynamic library by scanning the symbol table and demangling, at least on some platforms. Just don't look at how it's implemented. Unfortunately, such cursed things are not generally available for most languages, and if your compiler or OS isn't supported by Boost.DLL, too bad.

Why it matters

The fact anyone not using C has to either resort to cursed hacks or basically export all their symbols as C-style symbols with either bespoke mangling or no mangling means that we all wind up suffering with C's limitations.

It means we usually get no namespacing unless we do it ourselves. It means we can't hide our functions from C (which may be desirable, especially if we throw exceptions, because unwinding into C is a horrible idea). It means we have to contort ourselves into the C way of doing things, instead of expressing ourselves in our language's native tongue.

There's a spectre haunting programming. The spectre of C. I wish we could do away with it.

So... why do so few seem to care?

🤷‍♀️

I figured runtime modularity would be a killer feature everyone would want, especially a way to do it in a natural way for the language, instead of dropping to C or changing how our functions are exported.

But it seems not.

And it's saddening.

We should be able to do this! Something as simple as runtime modularity, which is far from a new concept, shouldn't be so hard or so hacky.

And what would fix this?

A better ABI where we can express our language's concepts would be helpful but since that won't ever happen...

Literally exporting a way to mangle a name from a path.

It isn't a perfect solution, but it would be a solution. It would be better than just trying to pretend we're speaking C.

I implore programming language authors, library authors, anyone, please please please, I beg you, export your name mangling APIs in an accessible way.

— Elizabeth Ashford (Elizafox) Fedi (elsewhere): @Elizafox@social.treehouse.systems Tip jar: PayPal || CashApp || LiberaPay