Calling Convention Differences

You will ensure proper use of calling conventions like cdecl or stdcall when interfacing with specific binary artifacts.

In this lesson, you will learn how to explicitly specify calling conventions in Rust to ensure your FFI code interacts correctly with non-Rust binary artifacts.

The Problem of ABI Mismatch

When you call a function in a C or C++ library, the binary doesn't just look for a symbol name; it expects arguments to be placed in specific locations—either on the stack or in dedicated CPU registers—and it expects a specific party to "clean up" the stack after the function returns. If Rust uses its default ABI ("Rust") and the external library expects "stdcall" or "cdecl", the mismatch will lead to stack corruption, register clobbering, or immediate segmentation faults.

Specifying Conventions

Rust provides the extern keyword to specify the calling convention. By default, extern "C" uses the platform's standard C calling convention (typically cdecl on x86_64, or varying conventions on x86). However, if you are interfacing with older Windows APIs or specific firmware blobs, you might encounter stdcall.

Here is how you define these in your FFI block:

extern "stdcall" {
    // Used for Win32 API functions where the callee cleans the stack
    fn GetSystemTime(lpSystemTime: *mut SystemTime);
}

extern "C" {
    // The standard choice for cross-platform C interoperability
    fn printf(format: *const i8, ...);
}

Why It Matters

The calling convention dictates the "contract" between the caller and the callee. For example, in stdcall, the function being called is responsible for removing arguments from the stack. In cdecl, the caller is responsible. If you mismatch these, the stack pointer will drift further away from its valid position with every call, eventually pointing to invalid memory.

When you are interfacing with binary artifacts (like a pre-compiled .dll or .so), always consult the header files or documentation of that artifact to verify if it requires extern "stdcall", extern "fastcall", or the default extern "C". Rust’s type system cannot verify the calling convention of a library for you, so this is a crucial area where explicit instructions are required to maintain memory safety.

Key Takeaway: You must explicitly specify the correct calling convention using extern "ABI" to match the target binary's expectations, or the resulting stack misalignment will lead to undefined behavior.