Skip to content Skip to sidebar Skip to footer
Showing posts with the label Bigdata

Python Pandas Error While Removing Extra White Space

I am trying to clean a column in data frame of extra white space using command. The data frame has … Read more Python Pandas Error While Removing Extra White Space

Matrix Multiplication Using Hdf5

I'm trying to multiplicate 2 big matrices with memory limit using hdf5 (pytables) but function … Read more Matrix Multiplication Using Hdf5

Split 10 Billion Line File Into 5,000 Files By Column Value In Perl Or Python

I have a 10 billion line tab-delimited file that I want to split into 5,000 sub-files, based on a c… Read more Split 10 Billion Line File Into 5,000 Files By Column Value In Perl Or Python

How To Predict Correctly In Sklearn Randomforestregressor?

I'm working on a big data project for my school project. My dataset looks like this: https://gi… Read more How To Predict Correctly In Sklearn Randomforestregressor?

Get A List Of Subdirectories

I know I can do this: data = sc.textFile('/hadoop_foo/a') data.count() 240 data = sc.textFi… Read more Get A List Of Subdirectories

Python Pandas Error While Removing Extra White Space

I am trying to clean a column in data frame of extra white space using command. The data frame has … Read more Python Pandas Error While Removing Extra White Space