Tyler Clemons » Programming

Weka, Ruby, Association Mining

Tyler — Sat, 15 May 2010 21:09:17 +0000

Weka is a collection of machine learning tools used for data mining. Weka is written in Java however it is possible to use Weka’s libraries inside Ruby. To do this, we must install the Java, Rjb, and of course obtain the Weka source code. In this example, I look at Associating Mining.

Refer to my previous example for setup instructions.

Running Weka

The following is an example of a frequent itemset finder.



require 'rjb'

#———————-
def asscm()

#Load Java Jar
dir = “./weka.jar”
#Have Rjb load the jar file, and pass Java command line arguments
Rjb::load(dir, jvmargs=["-Xmx1000M"])

#make k-means classifier
obj = Rjb::import(“weka.associations.Apriori”)
assc= obj.new

#load the data using Java and Weka
weather_src = Rjb::import(“java.io.FileReader”).new(“weather.nominal.arff”)
weather_data = Rjb::import(“weka.core.Instances”).new(weather_src)

#Find the frequent itemsets
assc.setCar(true) #mines for class association
assc.setLowerBoundMinSupport(0.25) #set minimum support
assc.buildAssociations(weather_data)
puts assc.toString

end

#———————-
asscm()

We first tell Rjb to load the specified classpath, for us it’s our Jar file. I passed command line arguments that specify the amount of RAM to use.

Rjb::import loads specific classes. These are relative to our classpath.

I call the constructor for the new classes by using the .new method from Ruby. Afterward, I can use the new object like any other Ruby object. The method names are as they are found in their Java source files. For an explanation of data type conversions, click here.

The dataset I used is different from the previous because we can’t use numerical values. The dataset is found inside the data folder downloaded with weka.jar.

As I said before, this can be done in JRuby. A great example can be found at this great blog post, which inspired my post.

Weka, Ruby, Decision Trees

Tyler — Sat, 15 May 2010 20:40:26 +0000

Refer to my previous example for setup instructions.

Running Weka

The following is an example of running a Decision Tree.



require 'rjb'

#———————-
def dtree()

#Load Java Jar
dir = “./weka.jar”
#Have Rjb load the jar file, and pass Java command line arguments
Rjb::load(dir, jvmargs=["-Xmx1000M"])

#make k-means classifier
obj = Rjb::import(“weka.classifiers.trees.J48″)
dtree = obj.new

#load the data using Java and Weka
labor_src = Rjb::import(“java.io.FileReader”).new(“labor.arff”)
labor_data = Rjb::import(“weka.core.Instances”).new(labor_src)

#set the class attribute, here it’s the last value, and then build the classifier
labor_data.setClassIndex(labor_data.numAttributes() – 1)
dtree.buildClassifier(labor_data)
puts dtree.toString

#examine the particular datapoints
points = labor_data.numInstances
points.times {|instance|

theclass = dtree.classifyInstance(labor_data.instance(instance))
point = labor_data.instance(instance).toString
puts “#{point} \t #{theclass}”

}

end

#———————-
dtree()

We first tell Rjb to load the specified classpath, for us it’s our Jar file. I passed command line arguments that specify the amount of RAM to use.

Rjb::import loads specific classes. These are relative to our classpath.

The dataset is found inside the data folder downloaded with weka.jar.

As I said before, this can be done in JRuby. A great example can be found at this great blog post, which inspired my post.

Weka and Ruby

Tyler — Thu, 06 May 2010 17:21:15 +0000

Setup

The first step is to install Java. That’s very simple, just follow this link to download Java. be sure to download the JDK.

Getting Rjb, which stands for Ruby Java Bridge, is a ruby-gem. You can either install the gem using the command line:



gem install rjb

or, we can download the gem from this link. The readme inside the gem is very easy to follow.

The last step in setup is obtaining Weka. On the download page of Weka, choose “Other platforms (Linux, etc.)“ Inside, we find a jar file called “weka.jar” This is what we need to continue.

Running Weka

The following is an example of running the SimpleKmeans classifier.



require 'rjb'

#———————-
def kmeans()

#Load Java Jar
dir = “./weka.jar”
#Have Rjb load the jar file, and pass Java command line arguments
Rjb::load(dir, jvmargs=["-Xmx1000M"])

#make k-means classifier
obj = Rjb::import(“weka.clusterers.SimpleKMeans”)
kmeans = obj.new

#load the data using Java and Weka
labor_src = Rjb::import(“java.io.FileReader”).new(“labor.arff”)
labor_data = Rjb::import(“weka.core.Instances”).new(labor_src)

#build the cluster and output the k-means data
kmeans.buildClusterer(labor_data)
puts kmeans.toString

#examine the particular datapoints
points = labor_data.numInstances
points.times {|instance|

cluster = kmeans.clusterInstance(labor_data.instance(instance))
point = labor_data.instance(instance).toString
puts “#{point} \t #{cluster}”

}

end

#———————-
kmeans()

This is a simple example of how to use K-means. We first tell Rjb to load the specified classpath, for us it’s our Jar file. I passed command line arguments that specify the amount of RAM to use.

Rjb::import loads specific classes. These are relative to our classpath.

The dataset is found inside the data folder downloaded with weka.jar.

I will probably look at the other classifiers and post some more example. As I said before, this can be done in JRuby. A great example can be found at this great blog post, which inspired my post.

Creating Adium Scripts (thats what she said)

Tyler — Sun, 11 Oct 2009 00:37:38 +0000

I was talking to Ben and saw another opening for the classic and overused “That’s what she said” joke. As I was typing it in, I realized I should just make it easier on myself and write a simple script to do it for me. So I did.

Creating it is very simple. First, we need to setup a file structure. Create a folder called whatshesaid.AdiumScripts. Then right click(CTLR+click), and click ’show package contents.’ Create a folder called Contents. Inside Contents, create a folder called Resources. The result:

->whatshesaid.AdiumScripts
–>Contents
—->Resources

The next step is to create the actual script. Enter the Resources folder. Open up the Script Editor from the Applications folder. If this is not installed, then just be prepared to save a file using the .scpt extension. Create a function like this:



on substitute() 

return "thats what she said"

end substitute

The on substitute() function will be called every time the script is used. Inside is an array of strings seperated by commas. Now the question is, how do we use the script?

Go up one level back to the Contents folder. Click HERE and drag the info.plist file into the directory. If you changed any of the defaults, make the appropriate changes.

And that’s it. Just click on the random.AdiumScripts icon and it should install. Type /she or %_she in a chatbox to test the script. That’s the simplest way to do it. There are some other XML properties that can be added but aren’t necessary for personal use. If you plan on publishing on Adium’s website, be sure to add any and all public standards.

You can add more substitutions by creating more script files and more entries into the XML tree. You can also edit the triggers by changing /she to any character sequence.

Your files will be installed in Library/Application Support/Adium 2.0/Scripts

Learning Python , C++ , and Data Structures

Tyler — Sun, 01 Feb 2009 03:11:17 +0000

Back in undergrad, the majority of our classes taught us sound software engineering principles using Python, C++, and sometimes Java. The first two were used quite extensively. The resource I used was invaluable to my success in mastering data structures, learning OOP, and granting me proper exposure to Python and C++. Data Structures and Algorithms Using Python and C++ is a great resource and I wanted to share it here. I believe it is great for beginners of both Python and C++ and those entering a data structures course. Click HERE to order a copy.

Exciting Changes to Ruby on Rails

Tyler — Thu, 22 Jan 2009 19:36:40 +0000

http://www.infoworld.com/article/09/01/12/02NF-ruby-on-rails-merb_1.html

Rails will add some new features and enhancements by merging with MERB. Should be interesting.

Creating Adium Scripts (iTunes)

Tyler — Sun, 31 Aug 2008 01:09:26 +0000

I use Adium and iTunes frequently. One of the cool things about the OS X environment is the integration between applications. For instance, Adium contacts can be linked to Address Book accounts. One of my personal favorites is the integration of Adium and iTunes. Adium can display details of what is currently playing on iTunes such as the name of the song and the artist. The trick is customizing these messages so that they can be presented as suitable away or status messages.

The best way to learn these concepts is to open up one. That is just what I did. I found a script that formatted iTunes data. I was not satisfied with all of its features so I modified it. I left credit to the original authors in the modified script.

First, we need to create a file structure. Create a folder called name.AdiumScripts. name can be anything you like, just make sure you don’t forget it. Then right click(CTLR+click), and click ’show package contents.’ Create a folder called Contents. Inside Contents, create a folder called Resources. The result:

->name.AdiumScripts
–>Contents
—->Resources

Enter the contents folder. Click HERE for the info.plist file that will direct Adium where to get your script. Place it in the contents folder.

The following is how the file looks:

CFBundleDevelopmentRegion
English
CFBundleExecutable
Adium
CFBundleGetInfoString
Nameit
CFBundleIdentifier
com.adiumx.iTunes.scripts
CFBundleInfoDictionaryVersion
1.2
CFBundleName
name
CFBundlePackageType
AdIM
Scripts

File
name
Keyword
%_name
Prefix Only

Requires User Interaction

Title
Message iTunes data

Set
Nameit

name corresponds to the name of your project from earlier. Under File, we specify the name of the script file that will be called each time %_name is called. Of course, you can change any of these. It is also possible to create more callable scripts by copying and pasting.

Now enter the Resources folder. Create a file called name, or whatever you may have changed name to, with the file extension .scpt. You can either use your favorite text editor or the Script Editor. This is where we will create our actual script that displays the data.

The following script has the following sample output:

Reading and listening to You Are The Pan
By John Williams
From Hook (Original Motion Picture Soundtrack).

The first line is shows my default action combined with the current song name. The next lines are the artist, and the album. If iTunes is not active, whether it is paused or not open, the default action will appear. In the above example, the default action is Reading. The modified script can be found by clicking HERE.

The code is pretty simple to follow so I won’t go into details here. It is very simple to add more formatting. It is just a matter of copying and pasting. The column names found in iTunes are pretty much the same as they are in AppleScript e.g. to access the genre of a track, just use the variable genre. Adding more substitutions, that is adding more features like %_name, is as simple as expanding the XML tree and placing the new script files in the same directory.

Like always, if you plan on creating a published add-on, be sure to adhere to any current standards. And that’s it. Just click on name.AdiumScripts to install.

Your files will be installed in Library/Application Support/Adium 2.0/Scripts

Creating Adium Scripts (Emoticons)

Tyler — Sat, 30 Aug 2008 14:41:53 +0000

So much fun open source applications are. Adium allows tons of customization. The best part is that it does not take an extensive knowledge of Objective-C, Adium’s primary language, to customize some if it’s features. One customizable feature is the emoticon set. Emoticons, Emotion Icons, are the combination of characters that resemble faces. In some web browsers and most chat programs, such as Firefox and Adium, emoticons are automatically translated into pictures. But there are so many emoticons that some clients do not include every combination. Good thing we can create our own.

In this example, we are only going to create one new emoticon. The process can be duplicated easily by copying and pasting. To start, create a folder called Name.adiumemoticonset. The name doesn’t really matter as long as you remember it later. Right click(CTLR+click) on your new folder, it should be a smiley face, and click ‘Show Package Contents.’ Open up the Property List Editor to create a .plist file. Name this file Emoticons.plist.

Click HERE for a shell of the .plist file.

That is all there is too customizing your own Adium “smiley set”. Yourpic.png is the element that controls which picture is called when the proper emoticon appears. In this case, the emoticon :) , most know it as , will cause Yourpic.png to show up. You can assign any number of emoticons to a particular picture by adding more nodes under Equivalents. Just copy and paste for more elements.

Yourpic.png, and any other .pngs, are stored in the same directory as Emoticons.plist. All of them are stored in the main directory of the package. Most .png files are around (19-21)x(19-21). You don’t have to conform to the size limits but you might not want large files crowding your chat window. Also note, the emoticons only appear on your end, so whomever you are talking with will only see their representation of :) and not Yourpic.png.

Your files will be installed in Library/Application Support/Adium 2.0/Emoticons

Creating Adium Scripts (Random Text)

Tyler — Sat, 30 Aug 2008 01:16:43 +0000

Adium is a free mesenger client for the Mac OS X. It allows users to connect to multiple chat services, such as AIM and ICQ, at once. It is open source, which means users are allowed to tinker with it as they see fit. With this extensibility, it becomes possible to type something like “/random” and generate a random message. I decided to take a quick look at what goes on under the hood and decided to give a go at creating some easy scripts.

One of the easiest ways to learn how to create an Adium script is to open up an exisiting script and look at how it works. Easy enough. First, we need to setup a file structure. Create a folder called random.AdiumScripts. Then right click(CTLR+click), and click ’show package contents.’ Create a folder called Contents. Inside Contents, create a folder called Resources. The result:

->random.AdiumScripts
–>Contents
—->Resources



on substitute()

set usethis to some item of {"A","B, "C"}

return "Random Line :   " & usethis

end substitute

The on substitute() function will be called every time the script is used. Inside is an array of strings seperated by commas. usethis is assigned to some random string inside the array and returned to Adium. Now the question is, how do we use the script?

Go up one level back to the Contents folder. Click HERE and drag the info.plist file into the directory. If you changed any of the defaults, make the appropriate changes.

And that’s it. Just click on the random.AdiumScripts icon and it should install. Type /random in a chatbox to test the script. That’s the simplest way to do it. There are some other XML properties that can be added but aren’t necessary for personal use. If you plan on publishing on Adium’s website, be sure to add any and all public standards.

You can add more substitutions by creating more script files and more entries into the XML tree. You can also edit the triggers by changing /random to any character sequence.

Your files will be installed in Library/Application Support/Adium 2.0/Scripts

iTunes Scripting with Python

Tyler — Fri, 29 Aug 2008 23:48:08 +0000

I ran across scripting for the Mac OSX awhile ago. I finally decided to take a look at it when I decided to play around with Adium’s open source code. The most dominant prescene I observed was Applescript. Applescript is nice and all, but Python is nicer. Scripting OSX applications with Python has been around for a bit. Using the easy to install appscript, scripting can be leveraged by the power of Python and Ruby.

One of my favorite applications on my Mac is iTunes, so its no surprise this is the first application I turned to for testing. To start, import the proper library:

from appscript import *

Afterward, we can create a handle to iTunes and tell it to play:

itunes = app('itunes')

itunes.play()

Running the above lines will open up the application called iTunes and execute the action play. It will play the current track and specific tracks can be passed as an argument. The action .stop() will stop iTunes. Of course, we can do so much more with appscript. For instance, we can create a playlist that only plays a specific album. I wrote a script that does just that.

The script can be found HERE in a zipped file.

It is documented and should be easy to follow. One more thing, it is also possible to assign values to each track as if they were normal data objects.