Tyler Clemons

FTP fun with Ruby

by on Jun.15, 2008, under FTP, Programming, Ruby

WordPress is pretty cool, save the editor. One of the coolest features is being able to switch the look and feel of my website without doing much extra work.

To do this, I just search Google for WordPress themes and about a dozen websites pop up with hundreds of themes. Cool. I download a couple of them, all free, and look for the easiest way to deploy them. However, the only way to do that, is to upload all of the files onto my web server via FTP.

No problem, FTP has multiple file commands… that don’t exactly work with directories 🙁 The simple solution is to find an FTP client for MacOS X and resort to a drag and drop method. Of course there exist some programs out there for free like Cyberduck, or Filezilla (but I use OS 10.4 so FileZilla is out), but since it’s the summer time, I decided to make my own. To be honest, I didn’t actually look for an FTP program before making my own lol! I fired up Ruby for a little fun.

It is a simple enough task to create an FTP instance in Ruby and to login. The fun part is deciding how to automate the placing of subdirectories because that feature is not supported by traditional FTP. It can be done iteratively, but that would be a nightmare for bookkeeping. For that method to work, you might have to employ a queue type structure that stores found directories. Or you could just use recursion which makes things a little bit simpler.

For those new to the idea of recursion, read about it HERE. Basically, we can call a function that opens a remote FTP directory, lets call it directory A, and iterates through it’s contents. When a directory, lets call it directory B, is found, instead of storing it away, we call the same function again on B. After the function operates on B, the stack returns back to the caller, in this case A, and continues through the contents of A. If any directories are found in A, B, or any subdirectories, the function just makes another recursive call. Of course, the recursive stack only goes so far but 99.99% of the time the stack won’t need to get close to the limit. Judging from the test I ran, Ruby 1.8.6 exhausted at a little over 6200 levels.

def test_depth(x)

puts x

test_depth(x+1)

end

test_depth(1)

Of course the length of the recursive stack depends on the amount of local variables declared.  I found that for every 4 local variables, and every 4 parameters passed into a function, you should expect to sacrifice 85 levels or 85 recursive calls.

If you aren’t familiar with Ruby FTP, click HERE. The FTP code, without error handling, some comments, and output statements, looks like:

def send_it()

#open local directory
Dir.chdir(@current_directory[0][-1])

#make a local copy of file list to reduce network calls
remote_list = @ftp.nlst()

Dir.entries(“.”).each { |thefile|

#be sure not to try and send the current and parent directory
if thefile != “.” and thefile != “..”

filename = File.basename(thefile)
#check if this is a file
if not File.directory?(thefile)

@ftp.putbinaryfile(thefile,thefile)

else

if not remote_list.index(filename)

@ftp.mkdir(filename)

end

#change directory and make recursive call
@ftp.chdir(filename)
@current_directory[0] << filename
@current_directory[1] << filename
send_it()

#move onto next file after sending directory
@ftp.chdir(“..”)
@current_directory[1].delete_at(-1)
Dir.chdir(“..”)
@current_directory[0].delete_at(-1)
end

end

}

end

That is the basics of the code for the send method. There are some bookkeepers that were created. The @ftp object holds the ftp instance and the @current_directory is a multi-dimensional array that stores where the ftp server directory is and where the local directory is. There is also a temporary object that stores the remote server’s contents called remote_list. Storing these locally allows us to reduce the amount of remote calls to the server because network communication ALWAYS has a varying time.  One of the nice things about sending is having the ability to check if a directory exist.  Ruby FTP can’t tell you if a remote object is a directory, or if the remote directory exist, automatically, it takes a little bit of work to find out.

Something else to note, it is not necessary to use the bookkeepers such as @current_directory.  It is possible to pass the local directory and remote directory as parameters into the recursive function instead of building a list to handle the current directories.  This type of design would be best suited for a standalone function that does not use the data found in the list but the method written above is used as a class method.  The following is a simple recursive method, assuming that the @ftp instance is already created:

def send_it(remote,local)

#open local and remote directories
Dir.chdir(local)

@ftp.chdir(remote)

#make a local copy of file list to reduce network calls
remote_list = @ftp.nlst()

Dir.entries(“.”).each { |thefile|

#be sure not to try and send the current and parent directory
if thefile != “.” and thefile != “..”

filename = File.basename(thefile)
#check if this is a file
if not File.directory?(thefile)

@ftp.putbinaryfile(thefile,thefile)

else

if not remote_list.index(filename)

@ftp.mkdir(filename)

end

#make recursive call
send_it(filename,filename)

#move back up to parent directory after sending directory

@ftp.chdir(“..”)
Dir.chdir(“..”)
end

end

}

end

FTP also fails miserably in deleting remote directories. To delete a remote directory, all of its contents must be deleted and that includes its subfolders and contents. Downloading a directory suffers from the same issues as sending and deleting. Using recursion, it is easy to do both methods. The code for each method, deleting and sending, is very similar to send_it and can be found in the link at the bottom of this page.

Of course, what would a program be without the use of classes. Wrapping these methods in classes allows me to run these operations over again easily and possibly build a GUI on top of it. The current product allows me to do stuff like this:

require ‘Multi_FTP’
testme = Multi_FTP.new()
testme.setup(“username”,”password”,”server”)
testme.go_get(“/local/destination”,”/remote/goal”)
testme.go_send(“/local/sendthis”,”/remote/tohere”)
testme.delete_directory(“/remote/delete_test”)
testme.close_ftp()

The code is pretty easy to follow. I create a Multi-FTP class. Then I pass in my credentials and connect to the server. What follows is a series of send and receives and a delete command.

Of course it isn’t perfect. It could use a GUI and other stuff. Adding a method that changes permissions is pretty simple.  It would mirror the delete method, but since I didn’t need it at the time I created the class, I never wrote it.  I didn’t test all of the error handling but looks golden.  It was fun making it though 🙂

I modified some of the code so that it will work on different types of FTP versions.  Some FTP servers don’t allow deleting folders, so I just delete the files in a directory and alert the user which folders could not be deleted.  I also created a function, called get_files_and_directories(), that addresses how to distinguish between a file and a directory on an FTP server.  This is great for checking if a directory exist on a remote server.

The class can be found by clicking HERE.

:, , , , , ,

5 Comments for this entry

Leave a Reply

Looking for something?

Use the form below to search the site: