Articles by Month
- July 2010 (2)
- June 2010 (3)
- May 2010 (5)
- February 2010 (4)
- September 2009 (1)
- July 2009 (2)
- June 2009 (2)
- April 2009 (8)
- January 2009 (3)
- November 2008 (6)
- June 2008 (1)
- May 2008 (5)
- April 2008 (1)
- March 2008 (3)
- February 2008 (5)
- January 2008 (10)
- June 2007 (1)
- May 2007 (1)
- April 2007 (2)
- October 2006 (2)
- September 2006 (4)
- August 2006 (2)
- February 2006 (1)
- December 2005 (1)
- September 2005 (4)
- August 2005 (5)
- July 2005 (2)
- June 2005 (2)
- May 2005 (5)
Links
-
Monthly Archives: April 2009
Parallel Distributed Computing Example
You may have seen article, Hadoop Example, AccessLogCountByHourOfDay. This is a distributed computing solution, using Hadoop. The purpose of this article is to dive into the theory behind this. To understand the power of distributed computing, we need to step … Continue reading
Share This Post On:
Hadoop Example, AccessLogCountByHourOfDay
Inspired by an article written by Tom White, AWS author and developer: “Running Hadoop MapReduce on Amazon EC2 and Amazon S3” Instead of minute of the week, this one does by Hour Of The Day. I just find this more … Continue reading
Share This Post On:
Inspiring MapReduce lectures by Google
Watched a set of 3 lectures run at Google, by Aaron Kimball, on MapReduce was inspiring to me. I feel like I have a much more solid grasp on MapReduce after watching these. I really liked how it started out … Continue reading
Share This Post On:
hadoop-0.18.3 Could not create the Java virtual machine
Installed hadoop on a VM, and needed to set the java heap size, -Xmx1000m, lower than the default 1000 to get it to work. I set the HADOOP_HEAPSIZE var in the conf/hadoop-env.sh dir to the lower value, but hadoop continued … Continue reading
Share This Post On:
Fuse mounting HDFS on CentOS 5
The first step is to get fuse installed. It’s not as simple as “yum install fuse” – it doesn’t ship with RHEL5/CentOS5. wget http://dag.wieers.com/rpm/packages/RPM-GPG-KEY.dag.txt rpm –import RPM-GPG-KEY.dag.txt rm RPM-GPG-KEY.dag.txt yum install yum-priorities wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.3.6-1.el5.rf.i386.rpm rpm -Uhv rpmforge-release-0.3.6-1.el5.rf.i386.rpm rm rpmforge-release-0.3.6-1.el5.rf.i386.rpm vim … Continue reading
Share This Post On:
Holy Smokes, Hadoop works with S3 directly!
bin/hadoop fs -put /path/to/source s3://<s3id>:<s3secret>@<bucket>/path/to/destination This is so cool. I’m guessing that I could also use S3 as my input or output directory for Map/Reduce jobs. Share This Post On:u=http://www.koopman.me/2009/04/holy-smokes-hadoop-works-with-s3-directly/&title=Holy+Smokes%2C+Hadoop+works+with+S3+directly%21″ title=”Share ‘Holy Smokes, Hadoop works with S3 directly!’ on BlueDot”>
Share This Post On:
Posted in Linux
Leave a comment
Hadoop Streaming with PHP
I’ve started my journey with Hadoop, and the first thing I wanted to try was Streaming, so I could run the mapper and reducer methods with PHP programs. The first thing I did was setup an alias: alias stream=’/usr/local/hadoop/bin/hadoop jar … Continue reading
Share This Post On:
Posted in Uncategorized
Leave a comment
IPv6 Presentation, Introduction to IPv6
I am doing a presentation on IPv6, at my company’s TechFest. This is a day event with keynote speakers, and break out sessions. The purpose of TechFest is to give the developers and engineers a break from their day to … Continue reading
Share This Post On:
Posted in Uncategorized
Leave a comment
