So i have been playing around with polars and smartcore but i have some trouble with converting from a Dataframe into a DenseMatrix.
In my code I'm reading in some simple csv data consisting of several columns containing numeric data.
This gets read into a Dataframe (polars) but a DenseMatrix (smartcore) wants as input (for instance) a Vec<Vec<f64>>.
So i do the conversion but it seems very troublesome, and non ideomatic.
// select some columns and concat the data into a single column containg lists of the column data.
// | Data |
// | [1, 2, 3] |
// | [1, 2, 3] |
// | [1, 2, 3] |
// | [1, 2, 3] |
let n = [concat_list([col("Pclass"), col("SibSp"), col("Parch")])?.alias("data")];
let df = df.clone().lazy().select(n).collect()?;
// iterate through the column and convert each number in the nested list into an f64
let data = df
.column("data")?
.list()?
.into_no_null_iter()
.map(|n| {
n.i64()?
.into_no_null_iter()
.map(|n| n as f64)
.collect::<Vec<f64>>()
})
.collect::<Vec<Vec<f64>>>();
let matrix = DenseMatrix::from_2d_vec(&data)
This solution above works, but it doesn't feel ideomatic, and the documentation for polars is outdated and lacking when it comes to rust. Does anyone have a better solution to this type of problem?