Friday, 1 October 2010

Control your EC2 instance with Google App Engine

Due to a lower budget I decided to move my applications to a micro instance.
To reduce costs even further, I thought about shutting down and restarting the instance on a daily basis. How to do that? Well, simple, just deploy a cron task to your App Engine application. Here is an example of a task file (cron.xml) :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
    "http://java.sun.com/dtd/web-app_2_3.dtd">
<cronentries>
  <cron>
    <url>/manageInstancesController.json?action=startInstance
    <description>Starting instance
    <schedule>every monday, tuesday, wednesday, thursday, friday 08:00
    <timezone>Europe/London
  </cron> 
  <cron>
    <url>/manageInstancesController.json?action=shutdownInstance
    <description>Shutting down instance
    <schedule>every monday, tuesday, wednesday, thursday, friday 20:10
    <timezone>Europe/London
  </cron> 
</cronentries>



As you can see, the cron daemon calls two url's, namely,

/manageInstancesController.json?action=startInstance

and

/manageInstancesController.json?action=shutdownInstance.


We will need to implement the calls from the Spring controller to your EC2 account.
Beforehand, make sure you already have an active EC2 account and that you have an image you want to start. In my case I also need to attach an EBS volume to that instance.

For external connections, the GAE security model requires you to use the URLFetch utility. Hence The AWS API will not work for you. To connect to AWS, please download the following adapted AWS API jar here

Your web-inf/lib directory must contain the follwing libraries (they are all included in the distribution) :

appengine-api-1.0-sdk-1.3.5.jar
commons-codec-1.3.jar
commons-httpclient-3.0.1.jar
commons-logging-1.1.1.jar
jackson-core-asl-1.4.3.jar
stax-api-1.0.1.jar
stax-1.2.0.jar

Here is the implementation of the controller :

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.HashMap;
import java.util.Properties;
import java.util.Set;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.web.servlet.ModelAndView;
import org.springframework.web.servlet.mvc.multiaction.MultiActionController;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.ec2.AmazonEC2;
import com.amazonaws.services.ec2.AmazonEC2Client;
import com.amazonaws.services.ec2.model.DescribeInstancesResult;
import com.amazonaws.services.ec2.model.DescribeSecurityGroupsResult;
import com.amazonaws.services.ec2.model.DetachVolumeRequest;
import com.amazonaws.services.ec2.model.DisassociateAddressRequest;
import com.amazonaws.services.ec2.model.EbsInstanceBlockDevice;
import com.amazonaws.services.ec2.model.Instance;
import com.amazonaws.services.ec2.model.InstanceBlockDeviceMapping;
import com.amazonaws.services.ec2.model.InstanceType;
import com.amazonaws.services.ec2.model.Placement;
import com.amazonaws.services.ec2.model.Reservation;
import com.amazonaws.services.ec2.model.RunInstancesRequest;
import com.amazonaws.services.ec2.model.SecurityGroup;
import com.amazonaws.services.ec2.model.StopInstancesRequest;
import com.amazonaws.services.ec2.model.TerminateInstancesRequest;

public class ManageInstancesController extends MultiActionController {
 protected final Log logger = LogFactory.getLog(getClass());

    static AmazonEC2      ec2;
    static String publicIp = "yourIp";
    static String dataVolume = "yourVolume";
    static String imageId = "yourImage";
    static String availabilityZone = "eu-west-1a";
    static String accessKey = "yourAccessKey";
    static String secretKey = "yourSecretKey";
    static String keyName = "yourKeyName";

    private List getSecurityGroups(){
     List securityNames = new ArrayList();
     DescribeSecurityGroupsResult describeSecurityGroupsResult =  ec2.describeSecurityGroups();
  List securityGroups =  describeSecurityGroupsResult.getSecurityGroups();
  for (SecurityGroup securityGroup:securityGroups){
   securityNames.add(securityGroup.getGroupName());
  }
     return securityNames;
    }
    
 public ModelAndView startInstance(HttpServletRequest req,
   HttpServletResponse resp) throws Exception {
  logger.info("Logging to EC2 platform");  
        init();         
  logger.info("Logged to EC2 platform");

  RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
  runInstancesRequest.setMaxCount(1);
  runInstancesRequest.setMinCount(1);
  
  runInstancesRequest.setKeyName(keyName);
  Placement placement = new Placement();
  placement.setAvailabilityZone(availabilityZone);
  runInstancesRequest.setInstanceType("t1.micro");
  runInstancesRequest.setSecurityGroups(getSecurityGroups());
  runInstancesRequest.setPlacement(placement);
  runInstancesRequest.setImageId(imageId);
  ec2.runInstances(runInstancesRequest);
        return null;  
 }

 
 
    private static void init() throws Exception { 
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        Properties properties = new Properties();
        properties.setProperty("accessKey", accessKey);
        properties.setProperty("secretKey", secretKey);
        properties.save(output, "");
        ByteArrayInputStream input = new ByteArrayInputStream(output.toByteArray());        
        AWSCredentials credentials = new PropertiesCredentials(input);
        ec2 = new AmazonEC2Client(credentials);
        ec2.setEndpoint("https://eu-west-1.ec2.amazonaws.com");
    }
    
 public ModelAndView shutdownInstance(HttpServletRequest req,
   HttpServletResponse resp) throws Exception {
   logger.info("Logging to EC2 platform");  
         init();         
   logger.info("Logged to EC2 platform");
   
         try {
          
             DescribeInstancesResult describeInstancesRequest = ec2.describeInstances();
             List reservations = describeInstancesRequest.getReservations();
             Set instances = new HashSet();
             logger.info("You have " + reservations.size() + " Amazon EC2 reservation(s) running.");

             for (Reservation reservation : reservations) { 
              logger.info("You have reservation :" + reservation.getReservationId());
              instances.addAll(reservation.getInstances());
             }

             logger.info("You have " + instances.size() + " Amazon EC2 instance(s) running.");

             Map ebsMapping = new HashMap();
             for (Instance instance:instances){
              List blockList = instance.getBlockDeviceMappings();
              for (InstanceBlockDeviceMapping mapping:blockList){
               EbsInstanceBlockDevice device =  mapping.getEbs();
               String volume = device.getVolumeId();
               if (volume.equals(dataVolume))
                ebsMapping.put(volume, mapping.getDeviceName());
              }
              logger.info("Finding instance :" + instance.getInstanceId());
              logger.info("Image "+instance.getImageId());
              if ("running".equals(instance.getState().getName())){
               logger.info("Correct instance. Shutting down");    
               stopInstance(instance.getInstanceId());               
               releasePublicIP();
               detachAllDevices(instance.getInstanceId(),ebsMapping);
              } 

             }

             
             
         } catch (AmazonServiceException ase) {
          logger.error("Caught Exception: " + ase.getMessage());
          logger.error("Reponse Status Code: " + ase.getStatusCode());
          logger.error("Error Code: " + ase.getErrorCode());
          logger.error("Request ID: " + ase.getRequestId());
         }

  
        return null;  
 }
 
 private void stopInstance(String instanceId){
        TerminateInstancesRequest stopInstancesRequest = new TerminateInstancesRequest();
        List instanceIds = new ArrayList();
        instanceIds.add(instanceId);
        stopInstancesRequest.setInstanceIds(instanceIds);
        ec2.terminateInstances(stopInstancesRequest);
 }

 private void releasePublicIP(){
        DisassociateAddressRequest disassociateAddressRequest = new DisassociateAddressRequest();
        disassociateAddressRequest.setPublicIp(publicIp);
        ec2.disassociateAddress(disassociateAddressRequest);  
 }
 
 private void detachAllDevices(String instanceId,Map ebsMapping){
  for (Map.Entry entry:ebsMapping.entrySet()){
         DetachVolumeRequest detachRequest = new DetachVolumeRequest();
         detachRequest.setInstanceId(instanceId);
         detachRequest.setVolumeId(entry.getKey());
         detachRequest.setDevice(entry.getValue());
         ec2.detachVolume(detachRequest);    
  }
 }
 
 
 
}

Wednesday, 10 March 2010

Smartgwt autosugget / autocomplete with remote data

Want to embed an autocomplete control in your form. It's easy. Here are the steps you have to follow to get one working. In this example, we will have a control proposing several cities to choose from. On the server side, we will have to define a dao method to filter our collection based on a client criteria:


Server side :

Code snippet taken from AgendaEntryDAOImpl:



public List<City> findCities(final String cityName) {

         List<City> cities = getCities();
         Predicate predicate =  new Predicate(){
           public boolean evaluate(java.lang.Object object){
             City city = (City) object;
             if (city.getName().indexOf(cityName)!=-1)
                   return true;    
             else
                   return false;         
           }      
         };
  
         CollectionUtils.filter(getCities(), predicate); 
         return cities;
 }

 private List<City> getCities() {

         List<City> cities = new ArrayList();

         City city =  new City();
         city.setId((long)1);
         city.setName("London");
         cities.add(city);

         city =  new City();
         city.setId((long)2);
         city.setName("Paris");
         cities.add(city);

  
         city =  new City();
         city.setId((long)3);
         city.setName("Milan");
         cities.add(city);

  
         return cities;
 }


Define your controller, ManageAgendaController  :

public ModelAndView findCities(HttpServletRequest req,
   HttpServletResponse resp) throws Exception {
  
             String name = req.getParameter("name");
             if (name==null)
                   name = "";
             List cities = agendaEntryDAO.findCities(name);         
             Map model = new HashMap();        
             ModelAndView modelAndView = new ModelAndView();
             model.put("jsonResponse", cities);   
             modelAndView.addAllObjects(model);
             View jsonView = new JSONView();
             modelAndView.setView(jsonView);
             return modelAndView;  
 }




On your client, define two form fields, one being your autocomplete field and a hidden hidden input field. The hidden field will be set on selecting a value from the suggested values. You will have to define a handler on selecting a value:


public String getCitiesActionURL(){
  return "manageAgendaController.json?action=findCities";
}


public void loadItems(Reflection... dto) {

.....

final HiddenItem idCity = new HiddenItem("idCity");

  DataSource cityDatasource = getOptionDatasource("/"
    + getCitiesActionURL());
  final ComboBoxItem city = new ComboBoxItem() {
   protected Criteria getPickListFilterCriteria() {
    String name = (String) getValue();
    Criteria criteria = new Criteria("name", name);
    return criteria;
   }
  };  
  
  
  city.addChangeHandler(new com.smartgwt.client.widgets.form.fields.events.ChangeHandler(){   
   public void onChange(com.smartgwt.client.widgets.form.fields.events.ChangeEvent event){
    Object value = event.getValue();
    if (value != null) {
     String idCitystr = (String) value;
     idCity.setValue(idCitystr);
    }
   }
   
   
  });
  
  
  city.setName("city");
  city.setTitle("City");
  city.setDisplayField("name");
  city.setValueField("id");  
  city.setOptionDataSource(cityDatasource); 

........

}


Define the datasource, you must specify the mapping between the deserialized server stream arriving in JSON format and your client record object residing on the client. Hence define 'id' and 'name' attributes.

private DataSource getOptionDatasource(String url) {

    DataSource datasource = new DataSource();
    DataSourceTextField id = new DataSourceTextField("id", "Id");
    id.setPrimaryKey(true);
    DataSourceTextField label = new DataSourceTextField("name", "name");
    datasource.setFields(id, label);
    datasource.setDataFormat(DSDataFormat.JSON);
    datasource.setDataURL(url);
    return datasource;

}


If everything works ok, it should look this:





Download the application source code here

I have deployed a working example to App Engine here

About Me

My Photo