websockets - downer

just figured out why web sockets (HTML5) are'nt all the rage as yet.

though the html5 browsers are all compliant and ready:

1. the server sides are lagging behind pathetically. no standard apis for websocket servlets
have a look at tomcat7 reco, websocket code
http://svn.apache.org/viewvc/tomcat/tc7.0.x/trunk/webapps/examples/WEB-INF/classes/websocket/chat/ChatWebSocketServlet.java?view=markup

2. websocket protocol tho starts out as http and then upgrades to a full-duplex, most proxies might still not play ball, so most likely its not going to work across WANs. compliance here is inching its way. shucks.


I guess its back to long polling and COMET, yeah, at for coming year

The changing face of software development

Change is inevitable and nowhere is it most apparent then in the software technology space. Surely we all adapt to these changes that are quite the norm in software development.  Here is a little retrospective view at ways in which software development, as we know it, is changing.

Developers are writing less code
Increasingly, developers are getting persuaded to write less code and instead use relevant, good quality, well tested frameworks and tools. The availability of several good quality open source and commercial frameworks have contributed to this trend.

Frameworks like Spring, Apache Commons, are not only time tested, but also quality metrics about these frameworks are readily available in the public domain. Gone are the days, when developers had to contend with heavy weight black box frameworks, with suspect and opaque quality, where developers were at the mercy of the vendors technical support.

Also modern popular frameworks are also remarkably non-intrusive in terms of their usage with applications that we are developing. No longer are we stuck coupling our applications to frameworks, especially when we don't want to.This means that alternative frameworks can be used, as drop in replacements. Also this makes its easier to segregate our application code / business logic and isolate it from changing technology.

Right tool for the right job
Ever tried to take out a screw with a hammer.Painful, time consuming and not pretty, about sums it up. Same is true for technology, choosing, the right technology and framework to suit your requirement was always important, but has now become a critical factor in success.

Knowing the features, flexibility, non-functional characteristics, learning curves, of the plethora of frameworks available is one thing,but then having the good judgement to choose the right framework to suit requirements, is a very much in-demand skill, that is now an implicit expectation from all good developers and teams.

In the application development space, the ability to write tonnes of code, is no longer as lucrative as the ability to spot the right tools and frameworks to use, during development.

Read-Understand-Apply cycle is shortening
While the Read-Understand-Apply cycle was always shorter and sharper for those in software development as compared to other professions, with rapidly changing software technology, this cycle is getting even shorter.

Gone are the days, when reasonable time frames for doing end to end proof of concepts were in months. With cloud infrastructure and platform services and instant provisioning and ready made software stacks, pilots and prototypes, are more about integration, and reasonable time frames for implementations have shrunk drastically.

Skills, such as the ability to quickly understand new technologies and frameworks,and to integrate them in meaningful ways to realize business requirements, are very much in demand. Unfortunately, like in coding, there are no standard measures of how well a developer can understand, adopt and apply tools and technologies, to build applications.

Leverage the internet wide knowledge bases
Forget social media based surveys, the most prolific examples of using crowd sourcing, are the way developers are asking for and getting answers, for all their technical queries on the internet, using websites like stackoverflow.com

Developers are the ones who can best help other developers and internet is turning out to be an incredible platform for sharing, storing and looking up technical queries. Wikis, good old bulletin boards, discussion forums, email archives, you name it. Trouble shooting, using a huge knowledge base called the internet, is really changing the way we are developing software.

Every problem encountered, has a solution just a web search away. Before we get carried away, there are limitations and drawbacks to excessive use of such practices. But on the whole, the ability to use the internet knowledge base to our advantage when doing software development is unarguably, as very good to have skill.

I am often tempted to give candidates for developer interviews, a topic they know little about, and ask them to get meaningful information or a high level solution based on information available on the internet Just as an exercise to test their potential, to be able to use the vast internet knowledge base.


Listen to the end users
A very positive fallout of the widespread usage and adoption of Agile development practices is that, developers no longer rely on requirements document alone. They are having their sprint deliverables, validated  by product owners periodically. This lends to more frequent and involved interaction between end users and developers. Developers are now required to be more aware of business terms and vocabulary, business processes and business priorities. The communication skills that are required for these added responsibilities of the developer also need to be taken into account.


Conclusion
While we may be aware of most of these changes, they somehow never make it, into the way we recruit new developers, into our teams. Here is hoping we start using, some of the points highlighted above, when we are choosing teams for doing application development.

Please feel free to add to the list of points mentioned above, would like to have newer ideas and perspectives.


Yours truely,
ganesh.ghag@gmail.com

Summary of use cases, using hadoop in enterprise

• Need to analysis / summarize / query / store unstructured or semi-structured data. Example:
• logs
• sensor data
• emails
• blogs
• web content
• DOCs / PDFs
• images
• videos
• Ability to support multiple data sources that are producing very disparate and unstructured data
• Rate at which data is generated is very high, continuous and unpredictable ( say 1 TB per day or per cycle)
• Data to be analyzed is massively distributed. eg logs
• Not possible to intercept data being generated at single / known source
• Using traditional ETL batch processes to summarize data is too time consuming or impractical or expensive
• Moving all the big data to one storage area network (SAN) or ETL server becomes infeasible with big data volumes. 
• Even if you can move the data, processing it is slow, limited to SAN bandwidth, and often fails to meet batch processing windows.
• There is a need to run analytics on raw data
• Queries that will be run on raw data are not determinate and hence, criteria / parameters for summarizing data are not know upfront
• Huge amount of data needs to be retained on cheap commodity hardware
• Using expensive storage, used by RDBMS is not feasible
• To be continued

Demystifying AOP, Getting started with LTW (load time weaving)

Often, using aspectj AOP, especially with LTW (load time weaving) is shrouded in mystery.
Thought I would write up a little note about getting started with Aspectj AOP LTW.

here goes...all you would need is a simple eclipse java project looking like below.

We would need 

1. an aspect source file - MySimpleLoggerAspect.java
2. a sample service which will get AOP-ed - SampleService.java
3. a test class with main method - Tester.java
4. an Aspectj, LTW related config file - META-INF\aop.xml
5. aspectjrt-1.7.0.jar and aspectjweaver-1.7.0.jar in your project classpath

MySimpleLoggerAspect.java is the logging aspect. For further details about writing aspectj aspects please refer www.eclipse.org/aspectj/docs.php. Listing of the simple aspect in java is below.

----------------------------------------------------------------

package com.ghag.rnd.aspects.ltw;

import org.aspectj.lang.ProceedingJoinPoint;

import org.aspectj.lang.annotation.Around;

import org.aspectj.lang.annotation.Aspect;

import org.aspectj.lang.annotation.Pointcut;

@Aspect

public class MySimpleLoggerAspect {

@Pointcut("execution(* *(..))")

public void myTraceCall() {

}

@Around("com.ghag.rnd.aspects.ltw.MySimpleLoggerAspect.myTraceCall()")

public Object myTrace(ProceedingJoinPoint joinPoint) throws Throwable

{

System.out.println("myTrace:before call "

+joinPoint.getTarget().getClass().getName()

+"."+joinPoint.getSignature().getName());

Object retVal = null;

try

{

retVal = joinPoint.proceed();

}

finally

{

System.out.println("myTrace:after call "+

joinPoint.getTarget().getClass().getName()

+"."+joinPoint.getSignature().getName() + " retval=" +retVal);

}

return retVal;

}

}

SampleService.java is as easy as given below:

----------------------------------------------------------------

package com.ghag.rnd.aspects.sample;

public class SampleService {

public String doService(String in){

System.out.println("inside doService");

return in;

}

}

Tester.java listing is a few more lines of code:
----------------------------------------------------------------

package com.ghag.test;

import com.ghag.rnd.aspects.sample.SampleService;

public class Tester {

public static void main(String[] args) {

new SampleService().doService("Ganesh Ghag");

}

}

And finally the META-INF\aop.xml listing is simple a self explanatory, especially the package names ;-)
----------------------------------------------------------------

<aspectj>

<aspects>

<aspect name="com.ghag.rnd.aspects.ltw.MySimpleLoggerAspect" />

</aspects>

 

<!--  <weaver options="-verbose -debug -showWeaveInfo"> -->

<weaver>

<include within="com.ghag.rnd.aspects.sample.*" />

<include within="com.ghag.rnd.aspects.ltw.*" />

</weaver>

</aspectj>

Now when you run Tester.java, just ensure you have the following paramater supplied as JVM argument:

-javaagent:/your dev env local /path/to/aspectj\aspectjweaver-1.7.0.jar

Thats it folks, when you run, Tester.java, SampleService call will get AOP-ed and give following output:

----------------------------------------------------------------

myTrace:before call com.ghag.rnd.aspects.sample.SampleService.doService

inside doService

myTrace:after call com.ghag.rnd.aspects.sample.SampleService.doService retval=Ganesh Ghag


Getting started with AspectJ AOP with LTW is that easy, folks!