Amazon EC2 flexibility: replacing a root volume in an instance

Time for some Amazon EC2 love.

Currently, our main servers are running on Amazon EC2. This last week, for a reason still unknown, we ran into a lack of disk space problem. The solution is to replace the disk, which in normal circumstances could be really bad… But with EC2, after understanding a little better how it works, this was simple and quick. And application downtime could be reduced to a minimum – around 20 minutes if I remember correctly – but this depends totally on the volume size.

These are the steps I had to take:

  • Stop the EC2 instance;
  • Take a snapshot of its volume;
  • Create a new volume, based on that snapshot, but with the desired new size;
  • Detach the old volume from the instance, attach the new, bigger one;
  • Start the instance again;
  • Execute ‘resize2fs /dev/sda1’ to make the linux installation aware of the new space.

Just remember to do all of this in the same availability zone, you cannot attach a volume from one zone into an instance from another one. You can use the command ‘df’ to check if the resize was successful.

Finally, the last step was to re-associate the correct IP address, so that everything works again as if nothing had happened.

Posted in misc | Leave a comment

Matching a range of numbers in Scala

Hi everybody!

Yesterday I did a presentation about Scala at Globalcode. It was very nice, thank you all who attended!

Now, one of the things I liked the most was the final part when people started making questions. I must admit I didn’t have answers to some of them, and this post is about one of those.

The question was something like: “can I pattern match against a range of numbers, like, say, 1 to 10?”

The answer is yes, although with a little more code than I thought necessary – yet, the solution is simple. First, if we want to match a simple number, we could write something like this:

myNumber match {
  case 10 => println("a ten")
  case num: Int => println("a number")
  case _ => println("something else")
}

In terms of matching numbers, we have two cases here. In one of them, we match directly a ’10’. In the other, we match any number. But how do we match a range of numbers? By adding something extra in the ‘any number’ case:

myNumber match {
  case 10 => println("a ten")
  case num if 0 until 100 contains num => println("a number between 0 and 100")
  case _ => println("something else")
}

This ‘something extra’ is called a ‘guard’. Guards allow us to add more verifications to values we match. With this, we can have any kind of restriction in any matched value. Also, we could have a ‘case’ for numbers between 0 and 100, and another one for any other number, like this:

myNumber match {
  case 10 => println("a ten")
  case num if 0 until 100 contains num => println("a number between 0 and 100")
  case num: Int => println("a number")
  case _ => println("something else")
}

Finally, please mind the order of the cases. Notice how we go from the more specific numbers, to the broader matching. This is not optional. you cannot put the broader case before the other ones – the code won’t even compile. In summary, this won’t work:

myNumber match {
  case num: Int => println("a number")
  case 10 => println("a ten")
  case num if 0 until 100 contains num => println("a number between 0 and 100")
  case _ => println("something else")
}

Now lets talks about how this work. Like a lot of things in Scala, ‘0 until 100’ is not a special construct. It is implemented as library – which means you could have implemented this yourself. The steps that the scala compiler has to take to make this magic happen is something similar to the following:

  • 0 is an Int, so the compiler tries to find an ‘until’ method in the Int class, but it doesn’t exist;
  • the compiler finds out that the ‘until’ method exists in the ‘RichInt’ class;
  • it would be perfect if 0 was a RichInt… so the compiler now searches for a way to transform an Int in a RichInt;
  • luckily, it finds this: ‘implicit def intWrapper (x: Int) : RichInt’ in the Predef object;
  • 0 is then wrapped in a RichInt, and the compiler calls its until method, with ‘100’ as the parameter;
  • the ‘until’ call results in a Range object and the compiler calls its ‘contains’ method, passing ‘num’ as the parameter;
  • the result of ‘contains’ is a boolean, which will decide if the number matches or not which what we want.

That’s quite a lot of stuff happening, even though there is nothing too complex in there. To close, one more last bit of information. Predef is an object that contains a lot of helper implicit methods, which are in the classpath of any scala application by default. Very handy.

I would like to thank Henrique Prage, who was in the Scala presentation, and sent me this link this morning: http://stackoverflow.com/questions/1346127/can-a-range-be-matched-in-scala – it really helped me find out quickly the answer to this question =)

Posted in scala | Tagged , , , , , , | 1 Comment

sbt and maven

A few months ago I talked about my first impressions about sbt (simple build tool) here. My conclusion at the time was that NetBeans + Maven combination would suit me better. This changed a little bit, so I’m revisiting this topic to explain why.

First, I explored sbt a little more. I certainly still have a lot to learn, but I found out one thing that is making a lot of difference: you can use a maven pom file to handle your dependencies. This is great for a few reasons:

  • you can leverage your existing knowledge of pom files;
  • I’m not comfortable (yet) with the sbt own way of handing dependencies;
  • you can use tools that doesn’t support sbt directly – actually I don’t know of any that does that.

Lets elaborate some more. First, lets say you have a project that already uses maven. The first point means you can just type ‘sbt’, create an sbt project, and the tool will be able use the pom.xml file as a source for the project dependencies configuration. You will just have to use the sbt command ‘update’ after each time you change the pom file – which is very reasonable.

Second, the way sbt handles dependencies. Code. Basically, you configure the dependencies in a scala file, that gets compiled with the rest of the project. I don’t find this bad nor good. For now, I don’t have a strong opinion about this – but its still not comfortable.

Finally, the third point means you can use, say, NetBeans, with its great maven support, while using sbt as well. For a small personal project I’m currently working on, this is being very useful. My current development setup consists of NetBeans as the IDE, with sbt running in a console window in the ‘~test’ state. Every time I hit save on a source file inside NetBeans, sbt gets it, compiles and runs the tests. Yet, because I’m using an IDE instead of simple a code editor, a have full source code completion – which is really nice.

Of course this doesn’t work perfectly all the time – NetBeans gets confused from time to time and you might have to ignore some editor warnings. Nothing too bad though.

Posted in scala | Tagged , , , , , , | Leave a comment

Intro stuff on Scala

Hi!

This is a quick post to share two introductory materials about Scala.

The first one is a video with a presentation done by Martin Odersky, creator of the Scala language. Great one, really worth watching: http://www.youtube.com/watch?v=zqFryHC018k

The second is a prezi presentation I created for a Scala introduction presentation I did down here in Brazil. It is in portuguese, but I guess you can get a lot of the contents even if you don’t speak the language. You can find it here: http://prezi.com/dfn4kezo6yjr/introducao-a-scala/

I also have some notes I used during the presentation, which I’ll probably turn into some blog posts – which means, of course, more Scala materials are to come =)

Posted in scala | Tagged , , , , , , | 3 Comments

Scala Introduction at Globalcode

Just a quick update. Wednesday next week I’ll be doing a Scala Introduction presentation at Globacode, down here in São Paulo, Brazil (in portuguese, of course :P). If you are in the area, come and join us 🙂

More information can be found here (again, in portuguese).

A few days after the presentation I will probably post some material used in the presentation here. Stay tuned.

Posted in scala | Tagged , , | Leave a comment

A Scala script for processing files

This time, we will approach Scala through a different angle: that of a script file. If you see Scala solely as a ‘normal’ programming language, this should demonstrate that Scala offers more possibilities than you knew.

The code I’ll show bellow was not created only for this post. It was actually used to solve a small problem in a project. This means that, even if it is not as nice and beautiful as it could be, it did the job well.

First. lets describe what we are trying to do here. We had several text files that needed a certain pattern to be found and replaced by something else. Of course, this job could be done manually… but with tens of files, it would take a while… and would be very error prone. Another option would be to write a shell script to solve the problem. But we don’t have anyone in the team that are really good shell scripters.

So I took the opportunity to solve the problem in Scala, with a small script, that would scan all the files, one by one, search for the pattern and replace it with the new value.

To write a Scala script, you just type the commands directly in the script file, instead of putting them inside a class. So the code consist basically of some commands, followed by some function definitions that are used by these commands. Here is the source code, and some explanation after it:

import java.io._
import scala.util.matching.Regex
import Regex._

// entry point for the script execution
processDir(new File("."))

def processDir(dir: File) {
 for (f <- dir.listFiles) {
  if (f.isDirectory) {
   processDir(f)
  } else {
   val extension = f.getName.lastIndexOf('.') match {
    case -1 => None
    case x:Int => new Some(f.getName.substring(x + 1).toLowerCase)
   }

   if (List(Some("java"), Some("sql")) contains extension) {
    processFile(f)
   }
  }
 }
}

def processFile(file: File) {
 val fileContents = getFileContents(file)
 if (fileContents != null) {
  val regex = "\u[0-9a-fA-F]{4}".r
  val newFileContents = regex.replaceAllIn(fileContents, m => replaceUnicode(m.matched))

  if (fileContents != newFileContents) {
   val newFile = new File(file.getName + ".conv")
   val writer = new BufferedWriter(new FileWriter(newFile))

   writer.write(newFileContents, 0, newFileContents.length)
   writer.close

   file.delete
   newFile.renameTo(file)
  }
 }
}

def getFileContents(file: File) = {
 val reader = new BufferedReader(new FileReader(file))

 var line = reader.readLine
 var content = line
 while (line != null) {
  line = reader.readLine
  if (line != null) {
   content += 'n' + line
  }
 }

 reader.close
 content
}

def replaceUnicode(s: String) = {
 val unicode = s.substring(2)
 val unicodeChars = new Array[Char](1)
 unicodeChars(0) = Integer.parseInt(unicode, 16).asInstanceOf[Char]

 new String(unicodeChars)
}

First interesting thing to noticing is that we are leveraging a lot of already existent Java API knowledge – mainly File stuff from java.io. Besides that, the code is pretty simple, containing basically some function definition and passing, and some recursion here and there.

In line 6 we start the script execution by calling the function processDir on the current directory. An interesting part of this function is the pattern matching at the lines 13-16 where we get the current file extension, which we will use in the if statement that follows. There, we simply ignore files that have no extension (i.e. got defined as None), or are not .sql or .java.

For the correct files, processFile is then called. From here, all that happens is basic Java IO stuff, except for one interesting code on line 29.  Here, for each string that is matched by the regex, we replace it by a new version, generate by the replaceUnicode function. This is probably the most interesting line in the entire script.

Posted in scala | Tagged , , , , , , , | 4 Comments

Why I gave up Google App Engine for now

Google App Engine is nice. Cloud Computing is nice. But I’m giving up GAE for now, and bellow is why.

I (re)started to play around with Google App Engine about one or two weeks ago. Of course I read a lot about it in the past, and decided to play a little bit.

The first thing you have to do is to go to http://appengine.google.com/ and Sign Up. You will have to enter a mobile number and they will send you an SMS to validate your registration. The problems started here for me. It simply didn’t work for my number. After trying several times and waiting a few hours, I decided to use my Mom’s mobile – and a few seconds later, she received the message. It should not be that complicated, but at least now I could try GAE out.

Then, I installed the Eclipse plugin and created a sample project. This is the nicer part of all. Just click the “Deploy to Google App Engine” button in the tool bar, fill in your google credentials in the dialog that will appear, and your application will be uploaded and deployed. Really easy and impressive.

So far so good. So why did I gave up using it? If you read this blog for some time, you already know I’m really interested in Scala. So, what I really wanted to do was to deploy something using Scala to the cloud. Specifically, I wanted to deploy a Lift web application.

It turned out that, during some readings, I discovered that the main feature I was using with Lift doesn’t work with GAE: Ajax and (Ajax) Comet. GAE has  some restrictions that you can usually overcome, but there is one that hits this Lift feature pretty hard:  you cannot create Threads in a Google App Engine application. Usually, this is not a problem, but Lift’s implementation of Ajax uses Actors which, if I understood correctly, are implemented with Threads – and this make a lot of sense.

So, no GAE for me, at least for now. At least until I get bored with Lift :p

Posted in java, scala, web development | Tagged , , , , , , , | 1 Comment

Simple Build Tool for Scala

Recently I started looking at some alternatives for increasing productivity when writing code and building systems in Scala. One option that is getting some attention is SBT – Simple Build Tool. In some ways, it is similar to Spring Roo. It generates some skeleton code (not sure how much though) and has an interactive shell that allows us to execute some tasks like compile and building, and even running in an application server if you are building an web application – which is what I was testing.

After some trouble installing, you can simply type sbt in a terminal to start the interactive shell. If the current directory doesn’t have an sbt project created, it will ask you to create one. Then, you can use commands like ~jetty-run to start your web application and make it run with jetty.

One thing I noticed is that it uses maven internally a lot (or seems to…) – at least for downloading dependencies and storing them in the local maven repository. But differently from maven, it copies the libraries used by the project to the project folder and use them from there. Also, although I’m not sure what this means exactly, sbt has several commands that have maven equivalents.

The web application I was trying was the Lift Chat Demo which you can find here: http://liftweb.net/getting_started. It is a pre-made sbt project; after downloading the sample, you execute sbt in the command line and can start playing (they have maven and other tool’s versions as well). When running this application with ~jetty-run changes made to the html files are detected on the fly. And so are the changes made to the Scala code. This is nice, but doable with maven as well.

This last statement leads to this post’s conclusion: sbt is a nice tool, but I’ll stick to maven for now, for one main reason: IDE support. Maven is greatly integrated with NetBeans (and Eclipse as well, I suppose), and I couldn’t find a nice way to use sbt with any IDE. Since I’m NOT a vi programmer, this leaves me with no option for now.

As a last note, please keep in mind that I only took a quick look at the most basic features of sbt – they probably have a whole lot more to offer. It just happens that I’m ok with the NetBeans + Maven combination for now.

And finally, a link that helped me clearing some ideas: http://php.jglobal.com/blog/?p=363

EDIT: Paul Phillips corrected me in an email in the Bay Area Scala Enthusiasts list about sbt using maven. Well, it doesn’t. It uses only the maven repositories, which is absolutely reasonable. Still, it certainly learned a lot with what maven does – again, very reasonable.

Posted in scala | Tagged , , , , , , , | 1 Comment

N900 Trial

Hello!

After a big pause, I’m back =)

For this comeback I’ll be talking about something off topic, yet interesting: a high end mobile phone.

Out of the blue, I got a chance to try out the Nokia N900. In this post I’ll describe my impressions.

First of all, I own a Nokia N97. Compared to it, the N900’s touch screen feels more responsive. The general impression I got is that this device is more polished than the one I have.

One characteristic that will please some people and make others hate the mobile is that it feels more like a computer than like a phone. The ‘average’ user will hate, the geek user will love. It comes with a Linux version, Maemo, with a nice package manager that allows you to install a lot of stuff easily. But you can fall-back to the command line if you want.

N900

N900

It comes with a ‘stylus’ pen, which is very handy, again better than the N97 one – actually, it seems that the newer versions of the N97 doesn’t even include the pen. I liked using the N900’s pen better than my fingers actually, and mainly when I was playing Mahjongg with it – one of the few games I downloaded from the application manager.

Since I’m a Java developer, I had to try and install Java. Its a bit more trouble than normal, but doable. You install a full Java SE, not a Java ME implementation, which is nice. But the rather small screen won’t help you when you try to run applications developed for a full size monitor. Anyway, I didn’t find much use for it, but it works.

Some readers probably already know that I love podcasts. This means that I HAD to try to use the N900 as a podcatcher. It has a gPodder version that works nicely. Thumbs up, and once again it is much better than the option available on the N97.

One last consideration is regarding the language support. In summary, the language support in Symbian (the N97 is a Symbian device) seems crap to me. I never could get japanese stuff to display properly. With Maemo on the other hand, it worked out of the box. I finally could open japanese websites correctly.

There are some drawbacks as well. But there is only a couple of things that I really disliked in the device: its size and weight. It won’t fit in any pocket, and forget about your shirt’s pocket: it would be really annoying to carry it there.

Finally, I would like to thank womworld for the opportunity. I loved the device and I would buy one if it wasn’t expensive, even more here in Brazil. Also, I would like to apologize to them for the delay when returning the device. It happens that the Brazilian customs puts a lot of bureaucracy on us, so returning the device was a pain.

That’s all, next time I’ll be back on track with Java or Scala or something along those lines. See ya!

Posted in misc | Tagged , , , , , , | 4 Comments

Profissao Java 2010

Hi!

Talking again about events: Profissao Java 2010 (something like Java Profession 2010 in english). This event happened yesterday, Saturday, June 26th, 2010 (obviously :P). Was an one-day event, right to the point filled with information regarding a lot of different topics, from JavaEE to Digital TV to entrepreneurship.

Talking about market opportunities and the right frame of mind when approaching career decisions, Bruno Sousa suggested a few books that we should read:

Bruno Sousa and a few book suggetions

Bruno Sousa and a few book suggetions

One other topic covered by the event was mobile platforms. Including some Java ME, Android and, despite the fact of not being Java, iPhone. I liked it. Here is a picture of Helder da Rocha talking about iPhone development:

Helder da Rocha on iPhone development

Helder da Rocha on iPhone development

Overall, I liked the event. The technical parts were interesting but what I liked the most was the focus on entrepreneurship that a lot of the presentations had. Hearing how others are building their business in our market is always interesting and fun – and help us avoiding some mistakes, when our time comes. The event run a little bit late, having ended after 8 pm, and we left really tired, but it was worth it.

Congratulations to the Globalcode team for one more great event =)

Posted in java | Tagged , , , , , , , , | Leave a comment