How to read from a binary file and load the data into a 3d vector in C++?

Question

I currently have some .bin files, each of which contains a matrix of size AxBxC. I would like to load it into a 3d vector vector<vector<vector<float>>> in C++, but encountered some problems.

I tried to flatten the matrix and load it into a buffer first, but the size of the buffer was weird so I got stuck there.

My code was:

vector<vector<vector<float>>> load_3d_bin(const std::string& path){
    vector<vector<vector<float>>> r;
    std::ifstream binary_feat(path.c_str(),std::ios::in | std::ios::binary);
    std::vector<float> flattened_feat((std::istream_iterator<char>(binary_feat)),std::istream_iterator<char>());
    for(auto i: flattened_feat){
        int value = i;
    }
    std::cout<<flattened_feat.size()<<endl;

// code to be written

   return r;
}

Ideally r would be a 3d vector, but I am stuck at the iterator part as the .size() outputs differently from total length = AxBxC, which was what I expected...

Any solution or feedback? Thanks in advance!

--Edit:

Sorry for not being clear enough! In my original .bin files, the floats are encoded as 3d matrices, i.e., in each file they are in an array of AxBxC size. I would like to first flatten the original 3d matrices and then reshape the flattened vectors into 3d vectors of size 'AxBxC'.

I managed to do the flattening part myself as follows:

typedef vector<vector<vector<float>>> t_f
typedef vector<vector<float>> m_f

t_f load_3d_bin(const std::string& path){
    t_f r;
    std::ifstream binary_feat(path.c_str(), std::ios::in | std::ios::binary);
    std::vector<float> flattened_feat;
    float tmp = 0;
    binary_feat.read(reinterpret_cast<char*>(&tmp), sizeof(float));
    while(!binary_feat.eof()) {
        binary_feat.read(reinterpret_cast<char*>(&tmp), sizeof(float));
        flattened_feat.push_back(tmp); 
    }
    std::cout<<"file size:::"<<flattened_feat.size()<< std::endl;

# the reshaping code to be written

return r;
}

And now with the flattened vector I have, I tried to reshape it into a 3d vector, which is the # the reshaping code to be written part above:

t_f r(A, m_f(B));
for (int i=0; i<A; i++){
    for (int j=0; j<B; j++){
        for (int k=0; k<C; k++){
            r[i][j][k] = flattened_feat[(i*B+j)*C+k];
        }
    }
}

But I got segmentation fault... Any idea why I got the error and how to solve it? Or is it better to directly load the 3d matrices in .bin files to 3d vectors? Thanks!

Answer 1

Assuming your bin files just contain a flat concatenation of floats in binary format, in row-major order:

vector<vector<vector<float>>> load_3d_bin(const std::string& path){
  vector<vector<vector<float>>> ret(A, vector<vector<float>>(B));
  std::ifstream binary_feat(path.c_str(),std::ios::in | std::ios::binary);

  for (int i = 0; i < A; i++) {
    for (int j = 0; j < B; j++) {
      for (int k = 0; k < C; k++) {
        auto& row = ret[i][j].emplace_back(C);
        binary_feat.read(row.data(), row.size() * sizeof(float));
      }
    }
  }
  return ret;
}

Alternatively, you could use the fantastic armadillo library:

fcube c;
c.load(path.c_str(), raw_binary);
c.reshape(A, B, C);

Answer 2

In the end I managed to do this myself.

The solution is as follows:

typedef vector<vector<vector<float>>> t_f
typedef vector<vector<float>> m_f
typedef vector<float> v_f

t_f load_3d_bin(const std::string& path){
    std::ifstream binary_feat(path.c_str(), std::ios::in | std::ios::binary);
    std::vector<float> flattened_feat;
    float tmp = 0;
    binary_feat.read(reinterpret_cast<char*>(&tmp), sizeof(float));
    while(!binary_feat.eof()) {
        binary_feat.read(reinterpret_cast<char*>(&tmp), sizeof(float));
        flattened_feat.push_back(tmp); 
    }

    t_f r(A, m_f(B,v_f(C)));
    for (int i=0; i<A; i++){
        for (int j=0; j<B; j++){
            for (int k=0; k<C; k++){
                r[i][j][k] = flattened_feat[(i*B+j)*C+k];
            }
        }
    }

    return r;
}

The returned r is a 3d vector<vector<vector<float>>> of size AxBxC.