Malloc allocates more memory then needed when using soci::use

Question

While investigating a strange tcmalloc error, my colleague and I traced the error to one line of code:

soci::session db;
...
db << "INSERT INTO `public.tablename` (..., ..., textvalue) VALUES ('...', '...', :1);",
  soci::use(std::to_string(someStruct.getUint32_t()) + "." + std::to_string(someStruct.getUint32_t()));

Everything worked just fine, as we changed it to:

soci::session db;
...
std::string temp = std::to_string(someStruct.getUint32_t()) + "." + std::to_string(someStruct.getUint32_t());
db << "INSERT INTO `public.tablename` (..., ..., textvalue) VALUES ('...', '...', :1);",
  soci::use(temp);

We got the tcmalloc error in the first version because the first code snippet wanted to allocate about 1.8GB at the point where this string is assembled in the parameter list and there was not enough memory available on the corresponding system. After we outsourced the assembly of the string as seen in the second code snippet this error did not occur anymore. Obviously, this string build from 2 integers and a single dot does not need anything near 1.8GB.

I would be very grateful if anyone could explain to me what exactly is going wrong. We suspect it has something to do with template deduction related to some black soci-libary magic, but are not sure.

Answer 1

As Jarod42 mentions, the problem is related to soci::use in particular. As the Soci Doku mentions:

If the data provided by user comes from another temporary variable, it might be possible for the compiler to arrange them in a way that the user data will be destroyed before the statement will have its chance to execute, referencing objects that no longer exist.

Following this explanation comes a bad example (which is roughly my first, non-working code example), followed by something that resembles my second (correct) code example.

So, in the end, the first code section is a sample of how not to do it. The error I wrote came from accessing uninitialized garbage because one of the first two strings in the temporary object was already destroyed before the entire string could be built. Also something that confused me a lot when debugging: Building with Clang did not trigger the error and building with Gcc did. The doc explains that too, since the problem has to do with compiler-dependent optimizations.

If you use soci::use you should assemble temporary objects before calling the function and then just pass them. It's probably more readable anyway - but you have to know that it can be wrong.