VST3 SDK: Problematic construction of std::string member in ClassInfo object causes program to crash

796 views Asked by At

The Problem:

I'm trying to use the VST Hosting utilities included in the SDK to load a plugin. The code is as shown:

#include "vst/v3/Vst3CommonIncludes.h"

int main()
{
    std::string vst3_module_path = R"(C:\Program Files\Common Files\VST3\Kontakt.vst3)";
    std::string error;
    std::shared_ptr<Module> vst_module = Module::create(vst3_module_path, error);
    std::vector<ClassInfo> class_infos = vst_module->getFactory().classInfos();;

    assert(error.empty());
    assert(class_infos.size());

    ClassInfo plugin_info = class_infos[0]; //Crash
    //... load the plugin and do more things
    return 0;
}

where Vst3CommonIncludes.h just includes all the VST SDK headers from pluginterfaces/vst and public.sdk/source/vst 8

In my case, the SDK comes in source and cmake files to build them into a static library. So my code and SDK code share the same compiler.

VST SDK sources from Steinberg Media


Investigation Done:

My investigation showed that PluginFactory::classInfos() returned corrupted data, trying to assign from them causes std::bad_alloc since the size of std::string is not valid.

Definition of PluginFactory::classInfos() in VST SDK:

PluginFactory::ClassInfos PluginFactory::classInfos () const noexcept
{
    auto count = classCount ();
    Optional<FactoryInfo> factoryInfo;
    ClassInfos classes;
    classes.reserve (count);
    auto f3 = Steinberg::FUnknownPtr<Steinberg::IPluginFactory3> (factory);
    auto f2 = Steinberg::FUnknownPtr<Steinberg::IPluginFactory2> (factory);
    Steinberg::PClassInfo ci;
    Steinberg::PClassInfo2 ci2;
    Steinberg::PClassInfoW ci3;
    for (uint32_t i = 0; i < count; ++i)
    {
        if (f3 && f3->getClassInfoUnicode (i, &ci3) == Steinberg::kResultTrue)
//------------Unexpected behaviour here--------------------
            classes.emplace_back (ci3);                //--
//---------------------------------------------------------
        else if (f2 && f2->getClassInfo2 (i, &ci2) == Steinberg::kResultTrue)
            classes.emplace_back (ci2);
        else if (factory->getClassInfo (i, &ci) == Steinberg::kResultTrue)
            classes.emplace_back (ci);
        auto& classInfo = classes.back ();
        if (classInfo.vendor ().empty ())
        {
            if (!factoryInfo)
                factoryInfo = Optional<FactoryInfo> (info ());
            classInfo.get ().vendor = factoryInfo->vendor ();
        }
    }
    return classes;
}

After the in-place construction of the new ClassInfo element, ClassInfo::data::category and other std::string members (name, vendor, etc.) reads <NULL> in debugger.

Stepping into the constructor of std::string, I've found the this pointer during construction of data.category is NOT equal to &data.category, and was offset by 4 bytes.

&data.category = 0x 0000 009b 546f ed14
this (std::string constructor scope) = 0x 0000 009b 546f ed18
//Actual address varies but the offset remains the same

Thus the string object became corrupted and later crashes the program.

Additional Information:


std::string related experiment:

Also, experimenting with ClassInfo and its string members, I ran into this:

ClassInfo ci;
ci.get().category = "testCategory";                  //OK
const_cast<string&>(ci.category()) = "testCategory"; //Crash, Access violation at 0xFFFFFFFFFFFFFFFF

I think it's highly related to the problem, but I couldn't come up with an explanation.

C++ standard consistency:

I also added

#if __cplusplus != 201703L
#error
#endif

to every relevant file, so I'm sure they share the same STL implementation, the problem will still occur.

Reproducing Attempt:

I hoped to recreate a minimal scenario where VST SDK is not included, and with my own MimicClassInfo that resembles the structure of original ClassInfo in some aspects. The problem does not occur.


Compiler and SDK info:

MSVC 14.37.32822 , using C++17 standard. VST SDK 3.7.8 build 34 (2023-05-15)

2

There are 2 answers

0
Dmytro Ovdiienko On

I'm not quite sure if following is a reason of the issue you're experiencing, but I believe it worth to check it too. Who knows.

The issue I found a couple of years ago is that the size of the member function pointer in VS is different based on context:

  • In case if you take the size of the member function pointer for the forwarded class, you get 24 (on 64-bit platform)
  • In case if you take the size of the member function pointer for the declared class, you get 8

Because of that in the x86 assembly compiler used invalid offsets to read/write the data for the data members in the class where member function pointer is used. So, sometimes to read for example std::string field1_; compiler used offset 8, sometimes it used 32. So that my field had been initialized at address 8 and read from address 32 (which is a data of another field). To figure that out one had to read x86 assembly.

Following is a minimal example of the code to reproduce the issue. It is still actual on vs2022.

/// @file 2.cpp
class X ;

size_t foo() noexcept { return sizeof(void (X::*)()); }
/// @file 1.cpp

// cl 1.cpp 2.cpp

class X {};

size_t foo() noexcept;
size_t bar() noexcept { return sizeof(void (X::*)()); }

#include <iostream>

int main() {
    std::cout << foo() << "\n";
    std::cout << bar() << "\n";
}

If you compile and run this app, you get 24 and 8.

How can it affect your app. Let's assume you have struct X and struct Y which encapsulates the pointer to the member function of struct X.

/// @file X.h

#pragma once

struct X{};
/// @file Y.h

#pragma once

#include <string>

struct X;

struct Y {
    void (X:: * field1_)();
    std::string field2_;
};

And you have two translation units main.cpp and 2.cpp

/// @file 2.cpp

#include "X.h"
#include "Y.h"

#include <iostream>

void foo(Y&y) noexcept {
    std::cout << y.field2_ << "\n";
}
/// @file main.cpp

#include "Y.h"

void foo(Y& y);

int main()
{
    auto y = Y{};
    y.field2_ = "Hello world";
    foo(y);

}

Now if you compile and run this app, it will crash.

0
Dmytro Ovdiienko On

Another reason of such behaviour is different compiler flags (especially macros) used to build the C++ files.

For example if some class X conditionally (based on some macro) defines one more additional data member, that affects the data members layout. If you compile two C++ files one with the macro and another one without that macro, your application may crash if those two C++ translation units work on the same object of class X.

So your C++ code can be syntactically correct, but because of different compiler flags, linker may build buggy app.