The Digital Enterprise - Part 3 - Leveraging Social Media

Continuing, off our discussion from Part 2, about using social media data in the context of enterprise applications. Let first look at the big picture of how social media analytics can be used in the context of an enterprise.

We talked earlier about the idea of leveraging the fact that end users are increasing using social media accounts like Facebook to login-in to company websites. Such a login can provide a trigger for requesting the user for sharing his/her social media profile details. These details can then be stored off and used as for offline analytics, to infer more about the end user. Getting to know the end user better in this fashion can then be translated into, targeted ad content as well as personalization for the user, to suite his current and future needs.

Refer to the architecture diagram below, which I will walk through in the discussion below:

1. We see the end user / customer accessing a company (say Insurance Company's) website or mobile app
2. The user is prompted to login to the company website using social account like Facebook account
3. After a 3-legged OAUTH 2 based authentication success, the company web app receives the login success callback.
4. Inside the login success callback for say facebook, a quick async ajax call ie POST is made to the Social Media Analytics Package (SMAP), which hosts our analytics application
5. The entire user profile data (in this case Facbook user profile) is persisted in a document oriented database like MongoDB
6. Similarly user profile data from twitter, linked-in, google-plus, etc can also be stored in the same MongoDB. Periodically,batches can be run,which will merge such aggregated social data based on keys such as Name, DOB, email addresses etc. Over a period of time, the Insurance company should have aggregated data for several customers and potentials, who have willingly registered on their website.
7. Now the same Social Media Analytics Package (SMAP) application can act as a provider for Clients which query for additional information about prospects and customers. Such queries can be issued and responded to over REST interface. Thin client browsers as well as mobile apps can access user profile information over REST interfaces
8. The SMAP REST interfaces can also be used by existing within enterprise CRM services, which can combine this social media user data with in-house CRM data, to produce consolidated profiles of users and prospects

The above architecture and high level design can be used to effectively, leverage social media based user profile information, to augment enterprise's own CRM databases.

On a side note, the same SMAP and MongoDB database can also be used to store website-level analytics data from Google Analytics, Facebook Insights, etc. Such data can be aggregated across large time periods to analyse a variety of user's trends and habits.

Cheers!

Simplifying REST with MongoDB

REST and MongoDB seem to be a match made in heaven. JSON is the popular format for transferring data across REST interfaces using the omnipresent and easy to use HTTP. BSON the data storage format of MongoDB is also a superset of JSON and is capable of accepting JSON input.

The natural expectation, then, is how can I use JSON in REST to directly store and retrieve data into MongoDB.

MongoDB's mongod, when started with the --rest option provides a bare minimum REST interface for querying MongoDB, using prototype URL of format http://127.0.0.1:28017/mydatabase/mycollection/

More information can be found at http://docs.mongodb.org/ecosystem/tools/http-interfaces/ under heading Simple REST Interface

But, for REST based insert, update, upsert and delete, you need to front Mongo DB with one of the following third party servers:

REST Interfaces
DrowsyDromedary is a REST layer for MongoDB based on Ruby.
MongoDB Rest is an alpha REST interface to MongoDB that uses the MongoDB Node Native driver.
Mongodb Java REST server based on Jetty.
Kule is a customizable REST interface for MongoDB based on Python.

HTTP Interfaces
Sleepy Mongoose (Python)
Sleepy Mongoose is a full featured HTTP interface for MongoDB.

Often, using these upfront REST to mongo translators, means intoducing another moving part into your architecture and having to contend with issues like cross origin access. Also there is the requirement to customize the REST to MongoDB translation. Hence many prefer to write their own simple REST controllers which delegate calls to MongoDB.

Well, so did I and gradually realized that, the code can be simplified quite a bit.

With the assumption that you have conventional java based REST controllers set up in your web application using say Jersey, here is the controller class you would have to write:

import javax.ws.rs.Path;

@Path("vehicles")
public class VehicleController extends MongoBaseController{
    @Override
    public String getEntityName() {
     return "vehicles";
    }

    @Override
    public String getKey() {
     return "vehicleId";
    }
}

Writing just this simple code, you can get, the following CRUD REST APIs, storing into your MongoDB database, since it will expose the following services:

at URL http://localhost:10080/<My Web App>/restful/vehicles
Http PUT for inserting vehicles in MongoDB
Http POST for updating and upserting vehicles in MongoDB
Http DELETE for deleting vehiles in MongoDB

The critical functionality as you might have guessed lies nicelt encapsulated in the base class MongoBaseController, which can to common to any number of entities that we want to CRUD from REST to MongoDB

package com.ghag.rnd.rest;


import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import com.ghag.mongodb.MongoCrudService;
import com.mongodb.BasicDBObject;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import com.mongodb.util.JSON;
 

public abstract class MongoBaseController {
    
    abstract public String getEntityName();
    abstract public String getKey(); 

    @PUT
    @Produces(MediaType.APPLICATION_JSON)
    @Consumes(MediaType.APPLICATION_JSON)
    public String insert(String entity){
     DBCollection coll = MongoCrudService.getConnection().getCollection(getEntityName());
     BasicDBObject row = (BasicDBObject)JSON.parse(entity);
  coll.insert(row);
  
  System.out.println("inserted entity "+row);
     return "SUCCESS_INSERT";
    }
    
    @POST
    @Produces(MediaType.APPLICATION_JSON)
    @Consumes(MediaType.APPLICATION_JSON)
    public String upsert(String entity){
     
     DBObject inputRow = (DBObject)JSON.parse(entity);
     DBCollection coll = MongoCrudService.getConnection().getCollection(getEntityName());
     
     DBObject src = coll.findOne(new BasicDBObject(getKey(), inputRow.get(getKey())));
     if(src == null)
      src = inputRow;
     
  coll.update(src, inputRow, true, false);
  
  System.out.println("upserted entity "+inputRow);
     return "SUCCESS_UPDATE";
    }
    
    @DELETE
    @Produces("text/json")
    @Consumes(MediaType.APPLICATION_JSON)
    public String delete(String entity){
  DBCollection coll = MongoCrudService.getConnection().getCollection(getEntityName());
  DBObject inputRow = (DBObject)JSON.parse(entity);
  DBObject src = coll.findOne(new BasicDBObject(getKey(), inputRow.get(getKey())));
  coll.remove(src);
  
  System.out.println("removed entity "+src);
     return "SUCCESS_DELETE";
    }
    
    
}

So now, if you want to persist another entity say Book via REST and into MongoDB all you would need to write is a small class similar to

@Path("books")
public class BookController extends MongoBaseController{
    @Override
    public String getEntityName() {
     return "books";
    }

    @Override
    public String getKey() {
     return "bookId";
    }
}

Thats it!

The new controller itself is free from any details of the structure of the Book entity, like its fields, data types etc.

That is truely combining the power of REST and MongoDB.

Mucho Eleganto! dont you think?

Cheers!

Getting started with Mongo DB

Once you are suitably convinced you do need a document oriented database for persisting your data, you can get started with using mongo db, the very popular document oriented database of our times.

While mongo db is the most similar to RDBMS as compared to other NOSQL databases, but there are a few surprising things you should know about mongo.

To recount a few:

Objects and arrays of objects can be used interchangeably in mongo db insert syntaxes

hence following is quite valid

db.mytablecoll.insert({name: ’Ganesh Ghag’, dob: new Date(1973, 6, 18, 18, 18), loves[’grape’,’watermelon’], hometown: 'Thane', gender: ’m’});

Update without Set

In its simplest form, update takes 2 arguments: the selector (where) to use and what field to update with.
db.mytablecoll.update({name: ’Ganesh Ghag’}, {hometown: 'Thane West'})
will replace the entire object aka row with {hometown: 'Thane West'}

to selectively update only one attribute of the row, you will need to use Set
db.mytablecoll.update({name: ’Ganesh Ghag’},  {$set: { hometown: 'Thane West'} } )

Multiple Updates surprise

The final surprise update has to offer is that, by default, it’ll update a single document.
so the following will only update the first row fulfilling the update criteria
db.mytablecoll.update({name: ’Ganesh Ghag’},  {$set: { hometown: 'Thane West'} } )

to ensure upsert functionality you will need a 3rd parameter "upsert" as true as shown below
db.mytablecoll.update({name: ’Ganesh Ghag’},  {$set: { hometown: 'Thane West'} }, true )

and to ensure all possible rows in collection are updated you need to set a fourth parameter "multiple" as true
db.mytablecoll.update({name: ’Ganesh Ghag’},  {$set: { hometown: 'Thane West'} }, true, true )

There is no "join" syntax in mongo db

All many to one and many to many relations are modelled using arrays of objects or embedded documents

Transactions in mongo

Mongo DB does not support transactions in classical sense, but has operators for atomic operations like $where and also 2-phase commit based manual transaction modelling

Support for geo-spatial queries

Mongo DB has direct support for geo-spatial indexex. This allows you to store x and y coordinates within documents and then find documents that are $near a set of coordinates or $within a box or circle.


If above mongo DB surprises are enough to keep you awake, wondering, at night, please visit Little Mongo DB book for an introduction to mongo db.



Cheers!

The Digital Enterprise - Part 2 - Leveraging Social Media

In order to critically evaluate where social media can impart maximum value to the enterprise lets consider the following back drop of how social media is getting used by enterprises.

Most companies have portals / websites for selling their products or services. Most such portals already have a social media dimension embedded in it, mostly in the form of "social media bars".

These social media bars which allow the portal users / customers to "like" or "share" or "comment" certain parts of their portal/website on popular social media like Facebook, twitter, LinkedIn, YouTube, etc.

Customers can also like or follow, the company Facebook and twitter accounts, registering interest in these. Customers end up endorsing these products or services on their individual timelines or feeds, thereby generating good sales leads.

So far so good.

Moving beyond using social media as marketing tools

 In the last couple of years, there is also an increasing trend of allowing users to login to company websites or portals using their social media accounts, with an option of sharing the customers social meda account information with the company.

For example, as a customer I can choose to login to a website xyz.com using my Facebook credentials and then, I get a "permissions" screen asking me if I would like to share my data with xyz.com. Once I acknowledge, I can effectively login into xyz.com.

Enterprise CRM Data can be complemented with social media derived customer data

Most companies maintain a CRM database containing their customer's details. This CRM data can be complemented with customer data obtained from social media.

While the social media data might be viewed as untrusted or non-validated, often it can provide great insights for marketing to act upon,

For example, in the CRM context, capturing data such as "marital status" might be a no-no, but in a social context, users may be okay, sharing that data as part of their Facebook public information.

Also things like Facebook profiles, likes, interests, public tweets, twitter following, twitter lists, etc can be analysed for well-known brands etc, can give back a wealth of information about the lifestyle preferences for the customer. This information can further we used to cross-sell and up-sell to the customer.

The following image depicts a possible architecture for collecting social media customer data, for purpose of analytic at a later stage.

Personalization - Targeted ads / content, for individual customer on portal / website

Once the above data mined from social media is available for our consumption, one creative way of using that data is within customer portal or website or mobile app.

Customers can be categorized by variety of factors such age, marital status, sex, etc. This categorization or segmentation can then be used to push content, ads and functionality that is more relevant to the customer segment.

For example, customers who are male and below 25 years, can be shown targeted ads like 2 seater cars. Also the insurance or wealth management or banking plans relevant to specific customer segments can be shown up more prominently on the website.

Summary

Looking at the above discussions, it is fairly clear that a social media analytics framework or module is very much the need of the hour, for every enterprise which is customer centric (who is'nt?)


In the next, part 3, of this series, I will look into "how" some of the above ideas can be implemented using existing tools, frameworks and technologies. The solutions need to be easily integrated into any portal / website and also need to be portable and extensible.

The Digital Enterprise - Part 1 - Basics

All over the world, the way business is getting done, is changing at an unprecedented rate. The major disruptive forces, contributing to this change, are

• Social Media Adoption
• Big Data
• Mobile Devices
• Cloud

These four factors are being popularly referred to as "Digital" - as a concise way to describe the digital revolution that is sweeping enterprises world wide.

Let us briefly consider the factors, contributing to "Digital Enterprise".

Social Media

The exponential increase in social media adoption, at a personal level, in last few years, is fairly evident. But what is really driving relevance of social media, at the level of enterprises and businesses is the fact that businesses and enterprises are becoming completely "customer - centric". 

Ok. Good businesses have always been customer centric :) but till last decade, the channels for knowing your customers were quite limited and conventional. With huge social media adoption, now, the channels for understanding your customer, their preferences, their needs, their likes and dislikes, habits, etc are much more easily accessible.

Businesses are using social media to engage with customers and conversely, customers too are expecting, websites and softwares to get customized, personalized to their likes, preferences and needs.

Typical use cases include

Businesses getting noticed by customers in social media context >> Customers, logging into product's or service's social pages, to see who amongst their friends and acquaintances recommend the product >> Customers expressing interest through likes, get quotes, etc

We can broadly categorize the "digital" initiatives in this area as being related to

• Customer Analytics - insights into the customers themselves
• Engagement Analytics - insights about how effectively our engagement tools are working
• Brand Promotion / Marketing

Big Data

The volume, variety and velocity of data being generated and made accessible is at an unprecedented levels. In order to analyse this magnitude of data, conventional batch processes are going to be following woefully short.

The ability to analyse such structured, semi-structured and unstructured data, so as to be able to derive business value from it, is going to need new tools and technologies that are distributed and can scale linearly with increasing data volumes.

Big Data platforms are definitely going to play a critical role in enterprise level business intelligence.

Mobile Devices

The explosion in the usage of mobile devices like smart phones, tablets, netbooks, etc has clearly transformed the way business is being done. The @anywhere - @anytime demand is being placed on enterprises not only from their customers but also from their internal staff.

Due to the ubiquitous use of mobile devices, business processes are getting transformed and new use cases, based on modern device capabilities like the ability to receive instant notifications or broadcast geo-locations, are transforming business processes like never before.

Mobile apps are changing customer engagement models and customer expectations of support and service levels are also being redefined in the context of these advancements in technology.

Cloud

Finally, the emergence of cloud computing and related technologies could not have been at a more opportune time. For example, in order to support the thousands of mobiles devices and mobile apps, which are going to access your IT Services, extremely scalable, performant and reliable infrastructures are a need of the hour. 

Elastic scalability, 24x7 availability and operational efficiencies are required to host the business applications of the digital enterprise. These can only be provided with a cloud infrastructure. Public, private and hybrid clouds are playing their part to support the digital enterprise, without a doubt.

Summary

The need of the hour is for enterprises to think of all the above factors - mobility, social media, big data and cloud, that contribute to a digital enterprise holistically, rather than trying to implement any one strategy piece-meal.

In the ensuing series of articles, I will be taking a deep dive into areas of interest related to digital, while trying to firm up concepts, solutions, tools, frameworks and technical designs and implementations.

Cheers !

Web Apps Development - Clearing the mist

Web 2.0, Ajax, REST, HTML5, javascript, jquery, node.js, single page apps, backbone.js the list goes on. How are all of these relevant? how do they fit in? when to use what? I found myself asking all these questions and things kept on getting misty, till finally I decided to look at all these "trends" in a way they had naturally evolved. Here is a log of that realization...

HTML, servlets, JSPs, the grand daddies of web apps
Post the decline of windows based business applications, since the late nineties, the browser based web applications have served business requirements well. Typically server side technologies like servlets and JSPs/ HTMLs were the work horses of such applications. These web apps were very much page centric, with html based pages dishing out functionality, navigating to additional pages etc. The need for reducing complexity and controlling page flows centrally was naturally was felt

Enter JSF..
JSF with their server side component models and declarative flow logic were quite a hit. But navigating to pages over http at internet scales, meant a rather poor user experience as compared to say desktop applications. Could this pain be reduced if not removed...

Enter ajax...
Ajax brought with it the ability for browsers to asynchronously communicate with server sides. Partial pages could now be refreshed on demand, by using javascript to issue out ajax requests. The data gotten over an ajax request was often many times lesser than the entire html page going over the wire. As ajax got more popular and powerful libraries like jquery emerged, it became a very accepted thing to load the html, css, javascript etc in the browser and then merely get the dynamic portions i.e. data to be requested over ajax calls.Server sides also started catching up and providing interfaces that could be easily consumed by clients like HTML+javascript.

Enter REST...
Emergence of REST-ful architectures and REST webservices, only added fuel to this fire and increasingly mashups and web apps were consuming REST services and were getting more decoupled from the server sides

Emergence of mobile apps and devices...
The same REST services could be consumed not only by browser based clients but also by native apps like iOS and android. since all they needed was a capable HttpClient. Soon mobile apps were all the rage and it only a matter of time before people started comparing them with conventional web apps. Mobile apps, with their native code, seemed much more responsive and provided much better UX, as compared to browser based apps. Could we have a browser based app, that communicated with the server only when it needed data, rather than request the entire view (with data)?

Enter Single Page Apps 
In single page apps, all necessary code – HTML, JavaScript, and CSS – is retrieved with a single page load, or the appropriate resources are dynamically loaded and added to the page as necessary, usually in response to user actions. The page does not reload at any point in the process, nor does control transfer to another page. Single page apps, optimize traffic between client and server, by going to server only for data input or output.This results in the role of the web server evolving into a pure data API or web service. Also, the UX is markedly improved since, unlike in page centric apps, views in UI dont travel across the wire. The notable cons, until recently were browser history, SEO to name a few. Everything said and done, the client sides and server side(REST services) are distinct.

Enter backbone.js and javascript frameworks explosion
With UIs increasingly getting built with client side technologies like HTML/5, CSS3, javascript, etc the complexity started moving over onto the client sides as well. To manage this complexity of maintaining, views, models in sync, view navigation logic, validations etc there was a dire need of frameworks and thus MVVM, MVC frameworks like backbone.js have started emerging out.Try having a look at sencha extjs to understand one more flavor of a pure javascript single page app framework

Enter node.js
Since the server side increasingly started becoming  a mere provider of REST services aka data, there was a need for simplifying server sides as compared to Java EE servers and also with the intention of making the server sides more scalable. hence the emergence of frameworks like  node.js

Enter full-stack frameworks like meteor
With node.js its very much possible to use javascript on server side and as a result, transparent client/server communication is possible, no longer do we need to develop web apps as strictly in client and server silos, we can reuse code such as validations across client and server and also using transparent client-server bridges/pub-sub, it is possible to write client javascript code to persist to a NOSQL database like mongodb. This simplifies the web application stack by order of magnitudes


Though the above stacks have been mentioned in evolutionary order, each has its own relevance no doubt and rather than picking out a winner or eventual winner, my attempt is to raise awareness about these options, so that the right tool can be used for the right job.


Cheers!

TryStack - Evaluate a Private IAAS Cloud at no cost

IAAS (Infrastructure-As-A-Service) public clouds like AWS and Rackspace are getting a huge amount of attention, no doubt. In the enterprise itself, private clouds based on OpenStack or Apache Cloud Stack, are also becoming quite popular.

I am sure, many like me, want to try their hands at using an IAAS style private cloud, without having to go through the hassle of installing private cloud on premise or using a paid IAAS public cloud like AWS.

Well, there is a good option for evaluating a private cloud stack like OpenStack completely online. Its called Try Stack. It runs a managed openstack environment.Its sponsored by donations from cisco, ubuntu, HP etc who want to popularize IAAS cloud computing.

Heres a quick rundown of how you can start using the hosted IAAS private cloud TryStack:

• Go to http://trystack.org/ and register onto their facebook group. For me the approval happened overnight
• You will get an email confirming above registration
• Then onwards you can login to the private cloud admin console using your facebook credentials

Following steps will  be familiar to people who have worked with private IAAS clouds like openstack or cloudstack

• Create a new security group opening up whichever ports you would like to access like TCP 22 for ssh, ICMP for pinging your instance, TCP 8080 for webserver access etc
• Create a new public-private key pair and download the private key for authentication into your instance
• Create a new instance choosing 
• template like micro, small, medium etc, 
• which OS to use: ubuntu, cirros, windows, etc are available in trystack
• the security group 
• key pair you created earlier
• additional volumes if required

• Check the instance boot logs, in the console, to ensure the VM instance was created all right

• Finally note the public IP, allocated to your instance
• Make sure you can ping your instance over the internet
• Use putty-gen to convert the openstack's private key .pem to .ppk format
• Use putty to ssh into your instance IP using the ppk private key
• You can also use winscp to login in to your spanking new VM instance

One warning: Since trystack is an experimental sandbox area, every 24 hours, your existing instances will be forcibly purged. 

Using Trystack, to familiarize yourself with using a full-fledged private IAAS cloud like openstack, is quite a painless experience and its free as well, you don't need to sign up with your credit cards.

Cheers!

Grails 2.3 released - provides out of box support for REST

Lets face it, REST is the undisputed king of content integration. Almost every major product, framework, supports exposing its user-consumable services over REST. The burgeoning popularity of mobile apps and thin client technologies, is also driving the adoption of REST. Part of its success is the ease with which REST services can be consumed. All you need is an Http client, no matter if you are developing UIs in HTML5/javascript, java, PHP, Ruby, Python, Android, iOS, etc... all the platforms have mature Http client libraries that can consume REST services.

Grails 2.3 which was released a few days ago, now provides out of the box support for exposing grails services functionality over REST. Grails was always very good at providing an easy way to write CRUD services based on its GORM/Hibernate based server side. The only sore point was its proprietary server side presentation technology in the form of GSP, which did not decouple the UI controllers sufficiently from the views. You still had developers driving GSPs, and making frequent and radically changes to UI (almost as if it were a mashup), were difficult.

But now...Grails can be used to develop service layers of an application, very rapidly and same services can very easily be exposed as REST interfaces, which can be consumed by a variety of clients including mobile clients.

Heres how...Getting started, is so easy even a novice developer could do this.

You can start by downloading the latest Grails 2.3 release.
Go about creating a new grails app as usual
grails create-app sample1

Create a new domain class
grails create-domain-class contact

Now annotate the class as shown below in bold:

package sample1
import grails.rest.*

@Resource(uri='/contacts')
class Contact {
        static constraints = {  }
String firstName;
String mobile;
String email;
}

Thats it, now just run your grails app with 

grails run-app

Congratulations! You have just REST-ified your domain class Contact.

You can perform CRUD operations like create, read, update and delete on your domain class Contact using the following REST URLs.

To fetch list of all contacts from DB use  

Http GET to  http://localhost:8080/sample1/contacts.json

To fetch a particular Contact information given its id = 101, use

Http GET to  http://localhost:8080/sample1/contacts/101.json   (where 101 is the id of the Contact)

To add a new contact

Http POST to  http://localhost:8080/sample1/contacts.json

with request body being a json like {'firstName':'Ganesh Ghag','mobile':'7776661110','email':'none'}

To update an existing contact with id = 101

Http PUT to  http://localhost:8080/sample1/contacts/101

with request body being a json like {'firstName':'Ganesh Ghag Updated'}

To delete an existing contact with id = 101

Http DELETE  to http://localhost:8080/sample1/contacts/101

While invoking Http verbs like GET and POST is fairly simple from an HTML, to be able to invoke PUT and DELETE, we will use cURL as shown below.

The sample curl script below, creates a new conatct, updates it, deletes it, just like a good test script should.

rem inserting a new contact

curl -i -X POST -H "Content-Type: application/json" -d "{'firstName':'From CURL1','mobile':'7776661110','email':'none'}" http://localhost:8080/sample1/contacts.json

rem updating a contact

curl -i -X PUT -H "Content-Type: application/json" -d "{'firstName':'From CURL Update 1'}" http://localhost:8080/sample1/contacts/1

rem deleting a contact

curl -i -X DELETE http://localhost:8080/sample1/contacts/1

rem getting all contacts after use case execution

curl -i -H "Content-Type:application/json" http://localhost:8080/sample1/contacts.json

Kudos to the Grails team for giving the REST features out of the box.

For more details please visit http://grails.org/doc/2.3.x/guide/webServices.html#REST

Building REST-enabled CRUD applications has never been easier!

Cheers !

Basics - One Time Passwords and Two Factor Authentication

Two-factor authentication requires the use of two of the following three authentication factors.

• Something the user knows (e.g., password, PIN, pattern);
• Something the user has (e.g., ATM card, smart card, mobile phone); and
• Something the user is (e.g., biometric characteristic, such as a fingerprint).

A clear explanation can be found on wikipedia:  http://en.wikipedia.org/wiki/Two-factor_authentication

The most popular and easiest way of implementing second factor in authentication is the use of OTP (one time passwords).

A one-time password (OTP) is a password that is valid for only one login session or transaction.
The most important shortcoming that is addressed by OTPs is that, in contrast to static passwords, they are not vulnerable to replay attacks.

There are also different ways to make the user aware of the next OTP to use.

• Some systems use special electronic security tokens that the user carries and that generate OTPs and show them using a small display. 
• Other systems consist of software that runs on the user's mobile phone (mobile apps)
• Yet other systems generate OTPs on the server-side and send them to the user using an out-of-band channel such as SMS messaging.

There are also well-known algos in this area such as HOTP = HMAC based OTP and TOTP = Time based OTP.

Some things to note:

• Use of hardware token genenators is more expensive
• Use of SMS channel requires the server sides to have capability to send SMS through reliable SMS gateways which may be paid options
• Use of mobile apps, often requires internet connectivity to the server

To further clarify the OTP options, I am providing a few self explanatory slides below:

--------------------xxx----------------------

-------------------xxx----------------------

Finally, there are several open source and paid providers of two factor auth and OTP solutions.Most of these require a custom authentication server as backend.

Open OTP     - http://www.rcdevs.com/products/openotp/

Lin OTP        - http://www.linotp.org/features.html

Phone factor  - https://www.phonefactor.com/how-it-works/overview.shtml

M-PIN          - http://www.certivox.com/m-pin/

and many more...

Also you can develop, a custom implementation using standard algorithms like HMAC for OTP

cheers!

Web Application Development with REST - Rapid and Easy

Increasing demands are being made by businesses to build custom web applications not only in record time, but also for these to be made available on a variety of devices like iPhones, iPads or android devices.

Using frameworks such as grails one can very quickly develop the server side of such web applications. With minor tweaks, a grails app can be exposed through REST controllers. These REST web services can be consumed by a variety of user interfaces like native mobile platforms, mobile web applications, single page thin client apps, HTML5 based web apps, etc. These user interfaces can also be integrated into portals such a Liferay, to complete the enterprise applications picture.

The above mentioned technologies and frameworks, relevant for rapid development of web and mobile applications, have been captured in the slide below. I will be periodically updating these slides, to raise awareness about contemporary frameworks that can be used for rapid web applications development.

(Note: this is just one good combination of frameworks with several alternatives on the tech landscape,  like Play instead of Grails, etc)

cheers!

Digitally Signed Documents - Simplified

Unless you have been living under a rock, you must have encountered, digitally signed documents. Increasingly all of us are using digital signatures in everyday life.

Ever wondered, what a digitally signed document means and how it works. Well here is a brief explanation through some pictures. Its from a lay man's point of view. If you are more interested in knowing the next level of technical details, have a look at the next slide which is technically more accurate.

Technically, more accurate depiction

Several free desktop softwares and online web applications allow you to digitally sign documents. for example, HelloSign, is a chrome plugin, which allows you to digitally sign documents, without leaving gmail.

Using REST with Grails - Quick and Dirty

Aim: To combine the RAD features of Grails with easy consumption of REST

Grails has been around for a bit and Grails 2.x is a great framework for rapid web application development. One of the things that might be a bit limiting is grails' use of server side gsp for the presentation layer. You know how everybody's grandmother, has an opinion about how the UI should be built, well...

That set me thinking, how about, if I wanted to leverage all the great feature of grails minus, its presentation layer, that is to say, how easily can I access grails server side from say HTML5/javascript framework or from android or iOS mobile app. Of course all that needs to be done, is to be able to invoke grails controllers from say javascript via REST calls.

So I started out with a default out of the box grails CRUD app with a POJO domain class called Item.

class Item {
static constraints = {    }
String name = "Item1";
String description = "Item1 Desc";
Date createdDate = new Date();

}

And I tried accessing the grails controllers from HTML/javascript/jquery. Seems you do it as easily as shown below:

For creating a new Item, all I need to do is a POST as seen below:
Notice the URL, HTTP verbs and $.param(myobj), which serializes, any javascript object attributes as HTTP request parameters

  //most of below object properties are same as grails domain POJO
var myobj = new Object();
myobj.createdDate = "date.struct";
myobj.createdDate_day = 13;
myobj.createdDate_month = 8;
myobj.createdDate_year = 2013;
myobj.description = "fromJSObject";
myobj.name = "fromJSObject";
myobj.create = "Create";

$.ajax({
 url: '/itemstore/item/save',
 type: 'POST',
 data: $.param(myobj),
 success: function(data) { alert('Insert was performed.');  }
});

On similar lines, we can do an update Item from jquery/javascript as follows

var myobj = new Object();

myobj.id = itemId;

myobj.version = 3;

myobj.createdDate = "date.struct";

myobj.createdDate_day = 13;

myobj.createdDate_month = 8;

myobj.createdDate_year = 2013;

myobj.description = "fromJSObjectUpdatedNew";

myobj.name = "fromJSObjectUpdatedNew";

myobj._action_update = "Update";

$.ajax({

 url: '/itemstore/item/edit/'+myobj.id,

 type: 'POST',

 data: $.param(myobj),

 success: function(data) { alert('Update was performed.');  }

});

Deleting an Item is as easy as:

var myobj = new Object();

myobj.id = itemId;

myobj._action_delete = "Delete";

$.ajax({

 url: '/itemstore/item/delete/'+myobj.id,

 type: 'POST',

 data: $.param(myobj),

 success: function(data) { alert('Delete was performed.');  }

});

The above way of invoking grails controllers is a bit of hack since, we did not modify the out of box or rather I should say "generated" controllers, which usually are geared up for server side invocation and have often have server side redirects. Sample grails controller server side code is shown below:

    def create() {

        [itemInstance: new Item(params)]

    }

    def save() {

        def itemInstance = new Item(params)

        if (!itemInstance.save(flush: true)) {

            render(view: "create", model: [itemInstance: itemInstance])

            return

        }

        flash.message = message(code: 'default.created.message', args: [message(code:        'item.label', default: 'Item'), itemInstance.id])

        redirect(action: "show", id: itemInstance.id)

    }

We were able to make do with shown jquery code because, we did not expect any return values from the controller invocations. Its always better to tweak the "generated" controller code, so that, it is oriented for REST, by commenting out the redirect and returning back either json objects as data or as validation messages.

But how about if we need some getters from the grails controller.Well, in such all you need to do is write your own little Grails controller with suitable methods.

Here I am showing the original controller code for a list() method returning all Items in database.

    def list(Integer max) {

        params.max = Math.min(max ?: 10, 100)

        [itemInstanceList: Item.list(params), itemInstanceTotal: Item.count()]

    }

And following is the slightly tweaked new method which will return JSON objects array

    def list(Integer max) {
        params.max = 100
render Item.list(params).encodeAsJSON()
    }

The significant difference is in calling the method groovy encodeAsJSON() and returning its result from the controller.

Next you have to add the following mapping to the UrlMappings.groovy file

"/restful/$controller/listall?"{

constraints {

         //maps the http verb GET to controller method list( )

action = [GET:"list"]  

}

}

Calling such a controller method from jquery/javascript is as easy as shown below:

$.getJSON("/itemstore/restful/item/listall", function(mydata) {

      

     // this will output valid JSON containing the array of Items we returned back

     console.log(mydata) 

});

Exposing grails controllers in such a fashion via REST or simply consuming them, in javascript, means you can have an alternative UI done in HTML5/javascript,, in a very short time.

Combining the RAD features of Grails with the ease of consumption of REST, is something truely valuable, wont you agree?

websockets - downer

just figured out why web sockets (HTML5) are'nt all the rage as yet.

though the html5 browsers are all compliant and ready:

1. the server sides are lagging behind pathetically. no standard apis for websocket servlets
have a look at tomcat7 reco, websocket code
http://svn.apache.org/viewvc/tomcat/tc7.0.x/trunk/webapps/examples/WEB-INF/classes/websocket/chat/ChatWebSocketServlet.java?view=markup

2. websocket protocol tho starts out as http and then upgrades to a full-duplex, most proxies might still not play ball, so most likely its not going to work across WANs. compliance here is inching its way. shucks.


I guess its back to long polling and COMET, yeah, at for coming year