If you write Lua C modules, there is a corner case surrounding
__gc metamethods) that you should be aware of. I
haven’t run into a public and prominent warning about this, so I
decided to write it up in this entry. I only discovered it myself
through trial and error.
Lua provides the
__gc metamethod for releasing resources associated
with a userdata. It is commonly used in C modules to free any C-level
objects or resources that were allocated by a type.
It turns out this will work nearly all of the time. But there is one very unusual corner case that can reliably cause gc() to run before call()! This program can trigger this behavior on both Lua 5.1 and Lua 5.2 (and LuaJIT):
Basically, any Lua code that runs inside a __gc metamethod can get access to a userdata that has already been finalized! This can crash your C extension if you don’t handle this case.
There are two main solutions to this problem:
- Clear the userdata’s metatable inside the finalizer. This will
ensure that the userdata fails any subsequent
luaL_checkudata()check later. The downside is that the error message for trying to call a method on a finalized value will be very unhelpful (something like “attempt to index field ‘?’ (a user data value)”)
- Set a “finalized” flag inside the finalizer, and check this flag
right after calling
luaL_checkudata(). For example, you could set the pointer to
NULLand check for this. This gives you the benefit of being able to return a custom error message like “attempted to call into dead object.”
Here is an example of what the second solution might look like:
For a bit more discussion about this, see this lua-l thread where I raised the issue.
The moral of the story is: anytime you are using
__gc from a C
module, you need to handle the case where the finalizer gets called
before other methods. Otherwise a user could SEGV your module.