Python Playground

I'm on a W8 machine, where I use Python (Anaconda distribution) to connect to Impala in our Had… Read more Connect To Impala Using Impyla Client With Kerberos Auth

What is the best way to create/write/update a file in remote HDFS from local python script? I am a… Read more Python Write To Hdfs File

I have installed, pip install thrift pip install PyHive pip install thrift-sasl and since pip ins… Read more Pyhive, Sqlalchemy Can Not Connect To Hadoop Sandbox

I have been running Hadoop MapReduce jobs by logging into SSH via PuTTy which requires that I enter… Read more Launch Hadoop Mapreduce Job Via Python Without Putty/ssh

I have a simple question to ask. I have been trying to execute HIVE queries from Python using impyl… Read more Why "importerror: No Module Named Builtins" Appears After Importing Hive From Pyhive Package?

I am very frustrated by Spark. An evening wasted thinking that I was doing something wrong but I ha… Read more Spark On Windows 10. 'files\spark\bin\..\jars""\' Is Not Recognized As An Internal Or External Command

I posted a similar question about an hour ago, but have since deleted it after realising I was aski… Read more Loading A Defaultdict In Hadoop Using Pickle And Sys.stdin

I'm currently taking a Big Data Class, and one of my projects is to run my Mapper/Reducer on a … Read more How To Run A Mrjob In A Local Hadoop Cluster With Hadoop Streaming?

My map task completes successfully and I can see the application logs, but reducer stays in pending… Read more Hadoop-streaming : Reduce Task In Pending State Says "no Room For Reduce Task."

I have a quick Hadoop Streaming question. If I'm using Python streaming and I have Python packa… Read more Managing Dependencies With Hadoop Streaming?

I'm using python and pyodbc module in particular to execute Hive queries on Hadoop. The portion… Read more Hive Parseexception In Drop Table Statement

I'm trying to setup Hadoop on my EC2 instance using this tutorial. I'm trying to setup the … Read more Ambari Server Setup: Oserror: [errno 2] No Such File Or Directory

Hey I'm fairly new to the world of Big Data. I came across this tutorial on http://musicmachin… Read more Running A Job Using Hadoop Streaming And Mrjob: Pipemapred.waitoutputthreads(): Subprocess Failed With Code 1

I know I can do this: data = sc.textFile('/hadoop_foo/a') data.count() 240 data = sc.textFi… Read more Get A List Of Subdirectories

I've seen the documentatio here, but I confess that I feel it rather lacking. I was wondering … Read more Python Udfs In Pig

How to get a list of files from hdfs (hadoop) directory using python script? I have tried with foll… Read more Get List Of Files From Hdfs (hadoop) Directory Using Python Script

My map task completes successfully and I can see the application logs, but reducer stays in pending… Read more Hadoop-streaming : Reduce Task In Pending State Says "No Room For Reduce Task."

I'm currently using Python 3 and would like to load a pickle file out of HDFS. from pywebhdfs.… Read more Unpickle File From HDFS

I have a simple question to ask. I have been trying to execute HIVE queries from Python using impyl… Read more Why "ImportError: No Module Named Builtins" Appears After Importing Hive From PyHive Package?