Tyler Clemons » Tyler

Weka, Ruby, Association Mining

Tyler — Sat, 15 May 2010 21:09:17 +0000

Weka is a collection of machine learning tools used for data mining. Weka is written in Java however it is possible to use Weka’s libraries inside Ruby. To do this, we must install the Java, Rjb, and of course obtain the Weka source code. In this example, I look at Associating Mining.

Refer to my previous example for setup instructions.

Running Weka

The following is an example of a frequent itemset finder.



require 'rjb'

#———————-
def asscm()

#Load Java Jar
dir = “./weka.jar”
#Have Rjb load the jar file, and pass Java command line arguments
Rjb::load(dir, jvmargs=["-Xmx1000M"])

#make k-means classifier
obj = Rjb::import(“weka.associations.Apriori”)
assc= obj.new

#load the data using Java and Weka
weather_src = Rjb::import(“java.io.FileReader”).new(“weather.nominal.arff”)
weather_data = Rjb::import(“weka.core.Instances”).new(weather_src)

#Find the frequent itemsets
assc.setCar(true) #mines for class association
assc.setLowerBoundMinSupport(0.25) #set minimum support
assc.buildAssociations(weather_data)
puts assc.toString

end

#———————-
asscm()

We first tell Rjb to load the specified classpath, for us it’s our Jar file. I passed command line arguments that specify the amount of RAM to use.

Rjb::import loads specific classes. These are relative to our classpath.

I call the constructor for the new classes by using the .new method from Ruby. Afterward, I can use the new object like any other Ruby object. The method names are as they are found in their Java source files. For an explanation of data type conversions, click here.

The dataset I used is different from the previous because we can’t use numerical values. The dataset is found inside the data folder downloaded with weka.jar.

As I said before, this can be done in JRuby. A great example can be found at this great blog post, which inspired my post.

Weka, Ruby, Decision Trees

Tyler — Sat, 15 May 2010 20:40:26 +0000

Refer to my previous example for setup instructions.

Running Weka

The following is an example of running a Decision Tree.



require 'rjb'

#———————-
def dtree()

#Load Java Jar
dir = “./weka.jar”
#Have Rjb load the jar file, and pass Java command line arguments
Rjb::load(dir, jvmargs=["-Xmx1000M"])

#make k-means classifier
obj = Rjb::import(“weka.classifiers.trees.J48″)
dtree = obj.new

#load the data using Java and Weka
labor_src = Rjb::import(“java.io.FileReader”).new(“labor.arff”)
labor_data = Rjb::import(“weka.core.Instances”).new(labor_src)

#set the class attribute, here it’s the last value, and then build the classifier
labor_data.setClassIndex(labor_data.numAttributes() – 1)
dtree.buildClassifier(labor_data)
puts dtree.toString

#examine the particular datapoints
points = labor_data.numInstances
points.times {|instance|

theclass = dtree.classifyInstance(labor_data.instance(instance))
point = labor_data.instance(instance).toString
puts “#{point} \t #{theclass}”

}

end

#———————-
dtree()

We first tell Rjb to load the specified classpath, for us it’s our Jar file. I passed command line arguments that specify the amount of RAM to use.

Rjb::import loads specific classes. These are relative to our classpath.

The dataset is found inside the data folder downloaded with weka.jar.

As I said before, this can be done in JRuby. A great example can be found at this great blog post, which inspired my post.

Youtube Time Markers

Tyler — Fri, 07 May 2010 18:58:51 +0000

I always forget how to do this, so I am going to put it on my blog If you have ever watched a youtube clip, and wanted to show your friends a particular snippet beginning at some specific time, you might send them the URL and tell them “Watch after 30seconds.” A better method of communication is to send the URL with a “time marker.”

Take the following youtube URL:

http://www.youtube.com/watch?v=xw1s3yXZLYc

If we add: #t=2m51s to the end of the url

http://www.youtube.com/watch?v=xw1s3yXZLYc#t=2m51s

It directs us to the video and starts playing at 2 minutes(2m) and 51 seconds(51s). Suppose the video is over one hour. We can do the following:

http://www.youtube.com/watch?v=7ImvlS8PLIo#t=1h2m30s

This tell us to start at 1 hour(1h) 2 minutes(2m) and 30 seconds(30s)

Weka and Ruby

Tyler — Thu, 06 May 2010 17:21:15 +0000

Setup

The first step is to install Java. That’s very simple, just follow this link to download Java. be sure to download the JDK.

Getting Rjb, which stands for Ruby Java Bridge, is a ruby-gem. You can either install the gem using the command line:



gem install rjb

or, we can download the gem from this link. The readme inside the gem is very easy to follow.

The last step in setup is obtaining Weka. On the download page of Weka, choose “Other platforms (Linux, etc.)“ Inside, we find a jar file called “weka.jar” This is what we need to continue.

Running Weka

The following is an example of running the SimpleKmeans classifier.



require 'rjb'

#———————-
def kmeans()

#Load Java Jar
dir = “./weka.jar”
#Have Rjb load the jar file, and pass Java command line arguments
Rjb::load(dir, jvmargs=["-Xmx1000M"])

#make k-means classifier
obj = Rjb::import(“weka.clusterers.SimpleKMeans”)
kmeans = obj.new

#load the data using Java and Weka
labor_src = Rjb::import(“java.io.FileReader”).new(“labor.arff”)
labor_data = Rjb::import(“weka.core.Instances”).new(labor_src)

#build the cluster and output the k-means data
kmeans.buildClusterer(labor_data)
puts kmeans.toString

#examine the particular datapoints
points = labor_data.numInstances
points.times {|instance|

cluster = kmeans.clusterInstance(labor_data.instance(instance))
point = labor_data.instance(instance).toString
puts “#{point} \t #{cluster}”

}

end

#———————-
kmeans()

This is a simple example of how to use K-means. We first tell Rjb to load the specified classpath, for us it’s our Jar file. I passed command line arguments that specify the amount of RAM to use.

Rjb::import loads specific classes. These are relative to our classpath.

The dataset is found inside the data folder downloaded with weka.jar.

I will probably look at the other classifiers and post some more example. As I said before, this can be done in JRuby. A great example can be found at this great blog post, which inspired my post.

The Ohio State Buckeyes: The 2010 Rose Bowl Champs!

Tyler — Sat, 02 Jan 2010 22:08:42 +0000

What a game for the Buckeyes! The 8th ranked Buckeyes made a great statement for their “slow” program and their “slow” conference with a 26-17 upset victory over the #7 Oregon Ducks. Great job Buckeyes!

The Good: TP, Posey, Sanz, and the the offensive line were great.

The Bad: OSU gave up four sacks to an undersized defensive line. I think part of the problem was TP’s indecisiveness which may have been due to his injury.

The Ugly: The Special Teams play for the Buckeyes was just plain awful. Come on Tress, this is supposed to be your special area!

Gamestats (From ESPN)

Picture courtesy of http://ohiostatebuckeyes.com/.

Creating Adium Scripts (thats what she said)

Tyler — Sun, 11 Oct 2009 00:37:38 +0000

I was talking to Ben and saw another opening for the classic and overused “That’s what she said” joke. As I was typing it in, I realized I should just make it easier on myself and write a simple script to do it for me. So I did.

Creating it is very simple. First, we need to setup a file structure. Create a folder called whatshesaid.AdiumScripts. Then right click(CTLR+click), and click ’show package contents.’ Create a folder called Contents. Inside Contents, create a folder called Resources. The result:

->whatshesaid.AdiumScripts
–>Contents
—->Resources

The next step is to create the actual script. Enter the Resources folder. Open up the Script Editor from the Applications folder. If this is not installed, then just be prepared to save a file using the .scpt extension. Create a function like this:



on substitute() 

return "thats what she said"

end substitute

The on substitute() function will be called every time the script is used. Inside is an array of strings seperated by commas. Now the question is, how do we use the script?

Go up one level back to the Contents folder. Click HERE and drag the info.plist file into the directory. If you changed any of the defaults, make the appropriate changes.

And that’s it. Just click on the random.AdiumScripts icon and it should install. Type /she or %_she in a chatbox to test the script. That’s the simplest way to do it. There are some other XML properties that can be added but aren’t necessary for personal use. If you plan on publishing on Adium’s website, be sure to add any and all public standards.

You can add more substitutions by creating more script files and more entries into the XML tree. You can also edit the triggers by changing /she to any character sequence.

Your files will be installed in Library/Application Support/Adium 2.0/Scripts

Kiva

Tyler — Thu, 01 Oct 2009 01:54:08 +0000

Ben showed me a great website recently. It’s called Kiva which can be found at http://www.kiva.org/. This website allows anyone to make a small loan to entrepreneurs in third world countries. The basic idea is to lend a small amount, or large amount, to an individual so that they can improve their business. They then pay the loan back. There isn’t any interest on the loan but in this situation, the benefits are worth more than interest. I suggest that everyone considers taking a look at Kiva. It’s a great opportunity to help those in need.

An apology to Alan Turing

Tyler — Tue, 18 Aug 2009 19:42:26 +0000

*It appears the petition may have worked*

http://www.number10.gov.uk/Page20571

Alan Turing is considered to be the father of modern computer science. Turing is also partly responsible for designing the “bombe,” a machine used to crack the German Enigma machine yielding invaluable German military Intelligence to the Allied forces.

Unfortunately, instead of being revered by his government for his services rendered, he was criminally prosecuted for being a homosexual because homosexuality was illegal in Great Britain in 1952. Having seen the end of his career and having undergone the subsequent drug treatment administered by the government to cure his homosexuality, he committed suicide via a cyanide induced apple.

Since then, the scientific community has embraced his achievements and ignored his lifestyle decision (as they should). With great pleasure, I have stumbled upon an online petition urging the British PM to apologize to Alan Turing. It is shame that only British citizens can sign it. This petition can be found:

http://petitions.number10.gov.uk/turing/

For more information about Alan Turing:

http://en.wikipedia.org/wiki/Alan_Turing

Lastly, what would this be without a plug for gay rights in today’s society:

http://www.aclu.org/lgbt/index.html

Website worth estimation

Tyler — Mon, 17 Aug 2009 22:05:49 +0000

Something that I stumbled on awhile back.

http://www.websiteoutlook.com/

It is no surprise that the most popular pages are adult websites.

OpenDNS

Tyler — Fri, 15 May 2009 23:15:28 +0000

I have been using this service for awhile now and I just thought I would formally recommend it. OpenDNS is usually a faster alternative to the default DNS provided by your ISP. It protects against phising automatically and can be adjusted to automatically block adult material aka porn. AND you can travel with your settings by creating an account.

Click here