SQL – Common Table Expressions (CTE) and HierarchyId

SQL Server 2008/R2/2012 offers hierarchyid data type that can be used for handling hierarchical data. In this blog I will try to take you down the rabbit hole and explore common table expressions and hierarchyid.

Quite often we have  to deal with hierarchical or recursive data. e.g.

Product Category (Entertainment Device)

     Product Sub-category (TV)

         Product Sub-sub-category (LCD)

              Product Sub-sub-sub-category (HD)

                      …..

I guess you get the idea.

 

Another example would be Employee – Manager relationship where every manager is an employee (but not every employee is a manager). Here’s a sample diagram:

image

So, how do we store this data in our table and then how do we query it? We have two options:

1. Common Table Expressions – CTE

This works in older versions of SQL Server 2005 and also the newer versions like 2008/R2, 2012. We use the traditional data types like int etc. to handle relationship.

Let’s first create the table so we can store the records and then we will query it.

   1:  CREATE TABLE dbo.Employee
   2:   ( 
   3:     EmployeeId int NOT NULL IDENTITY(1,1) PRIMARY KEY, 
   4:     Name varchar(50) NOT NULL, 
   5:     ManagerId int 
   6:  ) 
   7:  GO

 

Next step is to populate the table with the data (note: I am using same data that I have in my diagram)

   1:  INSERT INTO dbo.Employee (Name, ManagerId) 
   2:  VALUES 
   3:  ('CEO', NULL), 
   4:  ('Sales Manager', 1), 
   5:  ('Accounts Manager', 1), 
   6:  ('Senior Sales Person 1', 2), 
   7:  ('Senior Sales Person 2', 2), 
   8:  ('Senior Accounts Person 1', 3), 
   9:  ('Senior Accounts Person 2', 3), 
  10:  ('Junior Sales Person 1', 4), 
  11:  ('Junior Sales Person 2', 5), 
  12:  ('Junior Accounts Person 1', 6), 
  13:  ('Junior Accounts Person 2', 7) 

Now we are ready to use common table expressions to query the table.

   1:  --Finding descendants
   2:  WITH EmployeeDescendants(EmployeeId, Name, ManagerId) 
   3:  AS 
   4:  ( 
   5:  select EmployeeId, Name, ManagerId from dbo.Employee 
   6:  WHERE EmployeeId = 1 -- this is parameter
   7:  union all 
   8:  select m.EmployeeId, m.Name, m.ManagerId 
   9:  from EmployeeDescendants CTE,dbo.Employee m 
  10:  WHERE CTE.EmployeeId = m.ManagerId 
  11:  ) 
  12:  SELECT EmployeeId, Name, ManagerId 
  13:  FROM EmployeeDescendants 
  14:  ORDER BY EmployeeId
  15:   
  16:  --Finding ancestors
  17:   
  18:  WITH EmployeeAncestor(EmployeeId, Name, ManagerId) 
  19:  AS 
  20:  ( 
  21:  select EmployeeId, Name, ManagerId from dbo.Employee 
  22:  WHERE EmployeeId = 11 -- this is parameter
  23:  union all 
  24:  select m.EmployeeId, m.Name, m.ManagerId 
  25:  from EmployeeAncestor CTE,dbo.Employee m 
  26:  WHERE CTE.ManagerId = m.EmployeeId 
  27:  ) 
  28:  SELECT EmployeeId, Name, ManagerId 
  29:  FROM EmployeeAncestor 
  30:  ORDER BY EmployeeId ASC
  31:   

Common Table Expressions aren’t too bad, eh. You could easily convert the above script into a stored procedure and pass arguments to parameters.

Query Execution Plan for finding descendant nodes looks like this (Note: Plan for finding ancestors is similar):

image

Ouch, lots is going on in the query execution plan.

2. HierarchyId

This won’t work in SQL Server 2005. You must have newer versions like 2008/R2, 2012. We use the newer managed data type called “hierarchyid” to handle relationship.

Let’s first create the table so we can store the records and then we will query it.

   1:  CREATE TABLE [dbo].[EmployeeNew]
   2:  ( 
   3:      [EmployeeId] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY, 
   4:      [Name] [varchar](50) NOT NULL, 
   5:      [ManagerId] [int] NULL, 
   6:      [OrgNode] [hierarchyid] NULL
   7:  )
   8:   
   9:  GO 

Next step is to populate the table with sample records

   1:  INSERT INTO dbo.EmployeeNew(Name, ManagerId, OrgNode) 
   2:  VALUES 
   3:  ('CEO', NULL, hierarchyid::Parse('/')), 
   4:  ('Sales Manager', 1, hierarchyid::Parse('/1/')), 
   5:  ('Accounts Manager', 1, hierarchyid::Parse('/2/')), 
   6:  ('Senior Sales Person 1', 2, hierarchyid::Parse('/1/1/')), 
   7:  ('Senior Sales Person 2', 2, hierarchyid::Parse('/1/2/')), 
   8:  ('Senior Accounts Person 1', 3, hierarchyid::Parse('/2/1/')), 
   9:  ('Senior Accounts Person 2', 3, hierarchyid::Parse('/2/2/')), 
  10:  ('Junior Sales Person 1', 4, hierarchyid::Parse('/1/1/1/')), 
  11:  ('Junior Sales Person 2', 5, hierarchyid::Parse('/1/2/1/')), 
  12:  ('Junior Accounts Person 1', 6, hierarchyid::Parse('/2/1/1/')), 
  13:  ('Junior Accounts Person 2', 7, hierarchyid::Parse('/2/2/1/')) 
     

Time to test the table:

   1:  --display all records 
   2:  select orgnode.ToString() AS OrgNodeString,orgnode.GetLevel()  Level,* 
   3:  from dbo.EmployeeNew 
   4:   
   5:   
   6:  --getting root 
   7:  DECLARE @root hierarchyid = hierarchyid::GetRoot(); 
   8:  SELECT @root, @root.ToString()
   9:   
  10:  --employee we are interested in 
  11:  DECLARE @node hierarchyid 
  12:  SELECT @node = OrgNode FROM dbo.EmployeeNew 
  13:  WHERE Name = 'Sales Manager'; 
  14:  SELECT @node, @node.ToString()
  15:   
  16:  --get ancestors 
  17:  SELECT * FROM dbo.EmployeeNew WHERE @node.IsDescendantOf(orgnode) = 1;
  18:   
  19:  --get descendants 
  20:  SELECT * FROM dbo.EmployeeNew WHERE OrgNode.IsDescendantOf(@node) = 1;
  21:   
  22:  --get by level (junior employees) 
  23:  SELECT * FROM dbo.EmployeeNew 
  24:  WHERE OrgNode.GetLevel() = 3;

And the query execution plan for finding descendants looks like this (Note: query execution plan for finding ancestors is similar):

image

That query execution plan is sweet and simple, eh. But, don’t let it mislead you. Under the hood there’s still lots going on. Bulk of the work is getting performed by the managed functions like IsDescendantOf, GetLevel etc.

HierarchyId definitely reduces the lines of code required to query the hierarchical or recursive data.

 

Conclusion

Use hierarchyid if you have newer SQL Server like 2008 and onwards. It’s designed just to tackle hierarchical scenarios.

Use CTE if you have older SQL Server like 2005. However, CTE is a generic query pattern which means it can be used even when we don’t have hierarchical data.

1 Comment

Power View – Highlight, Filter and Slicer

SQL Server 2012 brings Power View (aka. Project Crescent) which supports highlight filter and slicers. They are pretty much similar to PowerPivot. Let’s discuss when these features should be used

Let’s take the following scenario where we have a table as well a chart on the same Power View report:

image

Highlight:

It let’s you focus on the interested data. If you highlight a table it won’t do much except for highlighting the cell in the table.

image

But, if you highlight a bar in the chart then it filters the table control automatically

image

Filter:

Filter can be utilized at individual control level e.g. table, matrix, card, chart etc. It cuts down on the data that’s being displayed on the Power View canvas. Filter at control level doesn’t effect other controls on the canvas.

Here’s the filter applied on table (Note: applying filter on chart will have similar effect):

image

One benefit of filter is that it has it’s own section (the right hand side window which can be closed or opened) where it gets displayed so it doesn’t consume real estate on the screen, well at least not all the time. You can choose to display filter pane whenever you require it.

Slicer:

Slicer is a filter but it’s scope is broader than a filter. Slicer filters data at the View level. One drawback of slicer is that it’s always displayed in the Power View canvas which means it reduces the real estate on the screen for controls that we can use.

Here we have a slicer on the right hand side and it filters all the controls on the canvas:

image

 

Summary:

  • Use highlight for showing areas of interest as well for filtering (especially chart control)
  • Use filter when you have to filter individual controls without effecting other controls
  • Use slicer when you have to filter everything on the canvas

No Comments

Power View – Power to the User

Power Builder, PowerPivot. Let me guess what word is common in these technologies ,“POWER”.  Quite a few products and technologies have used the word “Power” to give a sense of empowerment to the users. And, I guess they deserve to use the name as they have delivered the results. Now there’s another one to join the family and that’s Power View (aka. Project Crescent) in SQL Server 2012.

 

So far Power View seems pretty good. It does empower the end users. Here’s the requirements for using Power View:

  • SharePoint 2010 Enterprise edition with SP1
  • SQL Server 2012 PowerPivot (SharePoint must be configured as well)
  • Silverlight 5 (What?? No HTML 5 or AJAX?? Sad smile)

 

Yes, you read it right. It’s a Silverlight 5 based application and it won’t run on iOS. That’s where Microsoft should have offered AJAX alternative as well so those who don’t have Silverlight plug-in could use it as well. If you could just come over this one itsy bitsy drawback Power View is definitely an excellent self-servicing BI feature for business users.

As far as data source goes, Power View requires one of the following:

  • Tabular Model (New in SSAS 2012)
  • PowerPivot based Excel report (this could use OLTP, OLAP or Tabular Model)

Here are some of the graphical visualizations that we can create in Power View:

  • Charts (Horizontal and Vertical)

image

  • Scatter Chart (with Play Axis. Simply love the animation for showing breakdown of the data over period of time)

image

  • Charts (with Tiles)

image

 

  • Views (OK, this is not really a visualization since it’s available out-of-the-box but I just got to mention it since it’s soooooo good. Just like Excel has sheets concept, PowerPoint has slides concept the Power View has views concept. It lets you add multiple views of the data in the same report. You could say that it’s Power View’s equivalent to Sub-report in SSRS)

image

Here are the pros, cons and the personal verdict (it’s by no means an exhaustive list of pros and cons)

Pros:

  • Easy to create reports in a matter of minutes
  • No programming / SQL knowledge is required
  • Out-of-the-Box animations and visualizations (Play Axis stole my heart away)
  • Easy to filter reports at control level or at the report level

Cons:

  • Uses Silverlight plug-in
  • Yet to see any support for creating custom extensions or plug-ins to develop custom visualizations and animations

 

Personal Verdict:

Excellent reporting tool for business users and delivers well considering it’s the first iteration of Power View. Definitely looking forward to it’s future iterations.

2 Comments

Understanding Automatic Reference Counting (ARC)

ARC was introduced in XCode 4.2. Immediately, the technology became shrouded in an aura of mystery. Experts proclaimed that there is no mystery. The technology is completely deterministic. It may be deterministic, but, exactly how the system behaves under different situations is not very well documented. In this article, I will try to explain the basics through examples.

Consider the code:

@interface Address : NSObject {
    @public
    NSString *city;
}
@end

@implementation Address
- (Address*) init: (NSString*) c {
    city = c;

    return self;
}

- (void) dealloc {
    NSLog(@"Destroying address: %@", city);
}
@end

@interface Customer : NSObject {
    NSString *name;
    Address *addr;
}
@end

@implementation Customer
- (Customer*) init: (NSString*) n withAddress: (Address*) a {
    //Note 1: Automatically retain on assignment
    name = n;
    addr = a;

    return self;
}

- (void) dealloc {
    NSLog(@"Destroying: %@", name);
    //Note 2: All member variables are released automatically
}

@end

Customer* objectReturnTest() {
    NSString * n = [[NSString alloc] initWithString: @"Billy Bob"];
    Address * a = [[Address alloc] init: @"New York City"];       

    Customer *c = [[Customer alloc] init: n withAddress: a];

    //Note 3: ARC will put the returned object in autorelease pool.
    return c;
}

A couple of basic things to note here. As “Note 1”  says, when an object is assigned to a variable, a call to retain is made automatically. This increments the reference count. As “Note 2” says, when an object is destroyed, all member variable objects are released for you. You no longer have to do that from the dealloc method.

Finally, when a method returns a newly created object, ARC will put the returned object in an autorelease pool. This is stated in “Note 3”.

Now, let’s use the code.

int main (int argc, const char * argv[])
{
    NSString * n = [[NSString alloc] initWithString: @"Johnny Walker"];
    Address * a = [[Address alloc] init: @"Miami"];
    Customer *c = [[Customer alloc] init: n withAddress: a];

    NSLog(@"Before force release");
    c = nil; //Force a release
    NSLog(@"After force release");

    @autoreleasepool {

        Customer *c2 = objectReturnTest();

    }
    NSLog(@"After autorelease pool block.");

    return 0;
}

The log output from this code will be:

Before force release

Destroying: Johnny Walker

After force release

Destroying: Billy Bob

Destroying address: New York City

After autorelease pool block.

Destroying address: Miami

A couple of things to note here. See how force release works. We set a variable to nil. ARC immediately releases the reference count. This causes the Customer object “Johnny Walker” to get destroyed. But, the member Address object “Miami” doesn’t get destroyed. That is because the Address variable a still has a reference to that object. This Address object gets destroyed at the very end of the main method.

The object return test works as expected. Customer “Billy Bob” is put in auto release pool. At the end of the @autoreleasepool block, the pool is drained and the object is released.

Finally, we will now test for another common scenario. Objects may be created within a method that are not returned. Exactly when are they released and destroyed. The answer is, at the end of the method. Here is an example:

void temporaryObjectTest() {
    Address * a1 = [[Address alloc] init: @"Los Angeles"];
    Address * a2 = [[Address alloc] init: @"Seattle"];
}

We can call the method as follows:

NSLog(@"Before method call");
temporaryObjectTest();
NSLog(@"After method call");

The log output will be:

Before method call

Destroying address: Seattle

Destroying address: Los Angeles

After method call

These temporary objects are not put in autorelease pool. They are simply released at the end of the method. Only a new object returned from a method is autoreleased.

2 Comments

Custom HTML Buttons with CSS

The default <button> and <input type="submit"> tags work great but look super drab. Sometimes, you want jazz things up by adding images and stylized text. This can be easily done by converting a simple <span> into a button replacement.

image

First, we will define a class called "button".

<!DOCTYPE html>
<html>
<head>

<style type="text/css">
.button {
	border-style: outset;
	border-width: 3px;
	padding: 10px 10px 10px 10px;
	border-color: #ECECEC;
	background: #DADADA;
	cursor: default;
}

.button:active {
	border-style: inset;
}
</style>

Note the DOCTYPE declaration. We use the HTML5 syntax which makes a difference in IE. Things may be different if you use an old style DOCTYPE.

The key is to use the :active pseudo CSS selector. This selector matches when an element is clicked on. In normal state, we set the border-style to be outset. When clicked, the style becomes inset. That is the magic.

Then, all we have to do is use the styles from <span> elements.

<p>
Custom HTML buttons: 
<span class="button"><img align="center" src="http://cdn1.iconfinder.com/data/icons/freeapplication/png/24x24/New.png" />
Add Item</span>
<span class="button"><img align="center" src="http://cdn1.iconfinder.com/data/icons/icojoy/shadow/standart/png/24x24/001_39.png" />
Refresh view</span>
</p>

No Comments

New Overview of Java Technology Class Released

We have recently released a one day “Overview of the Java Technology Landscape” class, WA2015.

WA2015 Overview of Java Technology Landscape

This class is a brief overview of all that is “Java” in the current environment.  Particular focus is paid to a section covering some of the more recent changes to Java technologies to keep people on the cutting edge of what is happening so they can make appropriate strategic decisions in the planning of migrations of existing applications or development of new applications.

Our idea is that we will keep updating this course from time to time with new developments in Java so it can be something easy to offer to introduce new people to Java or to “catch up” for those that first encountered Java some time ago and may not have been able to follow all of the more recent developments.

No Comments

Advanced Web Service Class Released

Web services development has gone from an area that is “new” for most people to something that many people have experience with.  This does not mean that things are not changing though.  Many people know how to do “basic” web services, perhaps even applying security but in the current servers there are so many more things you can do.

Some of these newest features that not everyone is familiar with are:

  • WS-Addressing – Allows web services to enable an asynchronous programming model for responses that may come after a long delay
  • WS-ReliableMessaging – Allows web service clients and providers to ensure that a message was delivered and resend if needed
  • MTOM – A more efficient way to transmit binary data with a web service operation, often important when transmitting a file or similar usage
  • Configuring WS-Security with web service policies using WS-Policy
  • WS-AtomicTransaction – Allows the service implementation to communicate transactional qualities to the client and even allow the client to rollback the web service operation after it was invoked
  • WS-SecureConversation – This provides a much better performance of secure web service clients that send multiple messages over a short time

We have released a new 3 day class, WA1616 Advanced Web Services, to cover these advanced web service development areas.  This course also includes just a brief coverage of JAX-WS web service development in case developers are not aware of this relatively new Java framework for web service development.

This course would be appropriate for any developer that has some web service programming or XML experience but wants to dive deeper into these more advanced topics.  For those that might also want more coverage of the “basics” we could come up with a custom class that includes other topics before the advanced topics in this class.

Contact us to find out more about this exciting new course.

No Comments

Want a FREE WebSphere Eclipse IDE and development server with that?

UPDATE: We’ve posted some Java EE 6 development courses that use WebSphere and Eclipse to highlight this capability.  The best place to find these classes is in the WebSphere 8.0 Programming category.

UPDATE: If you are also interested in Spring tools for WebSphere development you might check out a related blog post – How about a FREE set of WebSphere and Spring Eclipse Development Tools?

UPDATE: Currently the versions of the Eclipse and WebSphere downloads are different than those detailed here.  The same process still works though and the important part is that if you want to run a WebSphere server from Eclipse (one of the main reasons to perform this setup) the Eclipse instance has to point to the IBM JVM. This is easiest to do by pointing it to the IBM JVM installed with the ‘WebSphere for Developers’ as outlined in this post.  Post questions if you have any as I try to keep up with the comments on this post.

ORIGINAL POST:

For the longest time the biggest complaint we’ve heard from developers who use WebSphere is “We can’t just use Eclipse!”.  In fact, the fact that the only “official” development support IBM would provide for WebSphere was Rational Application Developer created an entire market for MyEclipse Blue of people that didn’t want to pay for RAD but developed for WebSphere.  Even MyEclipse Blue is not free though, it is just cheaper than RAD.

So to solve this problem you do not know how many WebSphere shops I’ve seen that:

  • Do development with Eclipse
  • Manually add Java EE/WebSphere JARs to the classpath
  • Try to automate the process of exporting an EAR and deploying to WebSphere the best they can

Well now it is MUCH EASIER!  IBM has just released FREE Eclipse plugins to deploy/start/stop WebSphere v7.0 and v8.0 servers from the Eclipse ‘Servers’ view.  I bet you aren’t used to hearing IBM and “free” in the same sentence!  These are only available for WebSphere Application Server v7.0 & v8.0 (no WebSphere 6.1, WebSphere Portal or Process Server) but these are some of the most common WebSphere environments right now and in the near future.

All of this is done with the “WebSphere Application Server Developer Tools for Eclipse” in the “WASdev” community on IBM Developerworks.  Although you might see a lot of discussion about the WAS 8.5 “Liberty” profile which is an early program of some other exciting things they are doing, the WAS Developer Tools for Eclipse is something you can use NOW.

Setup Steps

Although you can certainly get some more info on how to use these new tools from the official announcement and the WASdev downloads page, I figure I would take it one step further and give you a complete set of instructions along with some screenshots for doing this on Windows.  It is really easy to do.

First of all I am going to assume for now that you are only using this for WebSphere v7.0 or v8.0 and not the “early program” (or Alpha) WAS 8.5 Liberty edition.  The reason this is important is the version of Eclipse to use.  For WAS 7 & 8 you will use Eclipse Helios.

UPDATE: It has been pointed out that there are 32-bit and 64-bit versions of both Eclipse and WebSphere.  The type you use for both has to be the same.  I would suggest the 32-bit since for development you don’t really need the expanded memory addresses of the 64-bit.  If for some reason you are using different architectures for the two products and getting an error probably the easiest way to fix it is to get the Eclipse version that matches the WebSphere you have installed.

  • First you will need to have a Java installation just like any other Eclipse environment even though we will just use it for the updates.  Later we will link Eclipse to the JVM installed with your local WebSphere server (this will be important).  You can get the Java download here although make sure it is not a ‘Java SE 7’ download as the Eclipse we will start with wouldn’t support that.
  • Next, go get the Helios release of the Eclipse for Java EE Developers.
  • Unzip the Eclipse download into an empty directory where you want Eclipse software to run from.  Even after we update with WebSphere tools it will run from this directory.
  • Run the ‘eclipse.exe’ file in the root of where you extracted Eclipse and open Eclipse in an empty “workspace” directory.  We won’t be creating projects yet but we still need to open a workspace to run the updates we need.
  • Make sure you are able to connect to the internet from within Eclipse.  If you have a proxy configuration this might mean going into the Eclipse preferences (Window –> Preferences) and the ‘General –> Network Connections’ section to setup network details.  A lot of times though Eclipse might be able to pick up your native internet settings so I would try this only if you have problems in the next steps.
  • Close the preferences if you had them open.
  • Once Eclipse opens, select ‘Help –> Check for Updates’ and wait for the update sites to be contacted.
  • Leave the default selected to update ‘Eclipse IDE for Java EE Developers’ and click the Next button.

image

  • Click the Next button again to confirm the update.
  • Accept the license terms and click the Finish button.
  • Once the installation of updates is complete, click the Restart Now button.
  • When prompted, open to the same workspace as before.
  • From the running Eclipse, select Help –> Eclipse Marketplace.
  • On the dialog that appears, leave Eclipse Marketplace selected and click the Next button.
  • On the Search tab that appears, fill in ‘WebSphere’ into the box and click the Find button.

image

  • Select the WebSphere v7.0 or v8.0 tools depending on which you want and click the Install button next to the one you want.  Note that the WebSphere v7.0 tools have an 8.0.4 in the text description but this is just the version of the Eclipse plugins.  If you want to do both you can, just select one and continue and then repeat the process after restarting Eclipse after the first set of tools is installed.

image

  • Leave the option to install your chosen tool version and click the Next button.

image

  • Accept the license terms and click the Finish button.
  • If prompted about installing unsigned content click the OK button to continue the installation.
  • Once the installation of updates is complete, click the Restart Now button.
  • When prompted, open to the same workspace as before.
  • After Eclipse opens, exit from Eclipse.

So that’s all you need right?  Not quite.  Besides the Eclipse tools you will also need a local WebSphere 7 or 8 test server. I know some of you are saying “darn, I knew this couldn’t be completely free!” Fear not! I promised the development environment would be completely free and it is because IBM has offered for a while now the WebSphere Application Server for Developers product, a COMPLETELY FREE version of WAS for development that is NOT a trial or expiring edition and is built from the same codebase as the production versions. The best link I’ve found to go straight to the WAS for Developers downloads is here.  For those of you with IBM Passport Advantage download abilities there is also a ‘WebSphere Application Server for Developers – Eclipse Tools Edition’ which has all of this but there is also a lot of stuff you probably don’t need.

We will also need to link the Eclipse environment to the Java JVM installed with the WebSphere server.  If you don’t do this you won’t be able to define a WebSphere server in Eclipse as the WebSphere Developer Tools will check that you are using an IBM JVM.

So just a few more steps and we will be set!

  • After downloading WebSphere Application Server for Developers from the link above or here go ahead and install it.  This will differ slightly depending on the version so it isn’t detailed here but should be pretty straightforward.  I always suggest NOT installing under the Windows ‘Program Files’ directories as on Windows Vista and above I have seen cases where Windows won’t let you save WebSphere configuration files so your configuration changes are mysteriously “lost”.
  • Once you have WAS installed make sure you have a “profile” configured.  This is the configuration that will serve as your test server.  You can run the ‘Profile Management Tool’ if a profile isn’t created during installation.  If you run the Profile Management Tool I always like to select the “Advanced” profile creation because you can use the “development template” for better development performance, make sure the server uses the default ports and NOT have the server run as a Windows service.  No matter how it is created know the name of the profile that you will use.
  • After WAS for Developers is installed go to the directory where you unzipped Eclipse.  Find the ‘eclipse.ini’ file and open it with a text editor.  The extension may be hidden and just list the file type as ‘Configuration Settings’.

image

  • Once you have the file open, add the following lines to the start of the file, substituting for <WAS Install Root> the directory you installed WAS at.  In the screenshot below my <WAS Install Root> was ‘C:\IBM\WebSphere\AppServer’.  Save the file when it is like below.  Make sure when you save it the file extension is not changed.

-vm
<WAS Install Root>\java\jre\bin\javaw.exe

image

UPDATE: As suggested in the comments, another way to alter the JVM used to start Eclipse is to add the ‘-vm <Path to JVM>’ syntax to the ‘Target’ property of a desktop shortcut that is used to launch Eclipse.  Setting the value in the ‘eclipse.ini’ file is to set the default and the command line option overrides this.  Your approach will probably depend on if you develop WebSphere and non-WebSphere projects with the same Eclipse install.

  • Once you have modified and saved the ‘eclipse.ini’ file, open Eclipse again to the same workspace.
  • Close the ‘Welcome’ page if it appears and find the ‘Servers’ view along the bottom.  This should open by default in the ‘Java EE’ perspective that should also be the default.
  • Right click in an empty area and select ‘New –> Server

image

  • Expand IBM and select your version of WAS and click the Next button.  This is where you would have errors if you hadn’t started Eclipse with the IBM JVM.

image

  • Since this is the first WebSphere server you defined, hit the Browse button and find the root directory of the WebSphere installation.
  • Once you have the WebSphere installation directory click the Next button.
  • On the next page make sure the correct profile name is listed in the drop-down.  Also make sure to supply security settings if you enabled security when the profile was created.
  • Once your settings are configured click the Finish button.
  • Start deploying projects from your workspace by right clicking the WebSphere server and selecting ‘Add and Remove..’, start and stop the server, even open the WebSphere Admin Console by going into the ‘Administration’ menu when right clicking!  In short, do all the things you’ve ever wanted to do to deploy and test on WebSphere from Eclipse!

image

Conclusion

So now you have a way to do WebSphere development with completely free tools without having to deploy remotely, hack the project classpath, come up with Ant scripts just to export and deploy, etc.  Obviously this opens up a whole new realm of possibilities for WebSphere development.  You will have much more freedom to construct the development environment that suits your project best but still make sure you are testing against WebSphere.

Since this is obviously a thing we know our own clients have been looking to do for quite a long time we plan on using this in several areas.  We will likely release a few Java EE training classes that use Eclipse and WebSphere 7.0 just to prove this is a completely capable development environment and let people get familiar with it.  We will also start using this quite heavily in the Java EE 6 on WebSphere 8.0 courses that will be posted soon.  We will also use this in our Spring 3 classes since one of our unique offerings has always been offering Spring development training on WebSphere.  Look for another blog post soon about the best development environment to use for Spring development for WebSphere.

So start dancing in the streets, we can finally develop and test WebSphere applications on “regular” Eclipse!

45 Comments

Overview of Java Technology Seminar Follow-up

Many thanks to all who attended the free one-day seminar where we looked at the range of Java technologies and where things are headed with some of the recent changes.  During the session there were many great questions and I decided it was just too difficult to follow up by email so I decided to create a blog post.

JSF

Clearly I think many of the questions that came up had to do with JSF.  I think this was a combination of people who were not yet familiar with JSF trying to understand the benefits in combination with people wanting to know about JSF 2 and how that might change the “should I use JSF?” question.

One thing that is definitely true is that people should no longer struggle with what I call “raw Servlet/JSP” which is using Servlet/JSP without any other framework.  There is just so much you have to do “manually” that you can’t concentrate on the business requirements of the application.  Perhaps this tendency is because the book “Core J2EE Patterns” was written before JSF was required by J2EE servers and tried to stay away from using any “de facto” standards like Struts.  If you look at that book it basically shows you how to write your own web framework built on Servlet/JSP.  Why do that when there are so many “pre-packaged” frameworks out there?  In fact, many of our Java EE 5 training classes focus on how to use technologies like JSF, EJB 3, and JPA to be more effective at programming Java EE applications and Java EE 6 will continue this trend.

  • What makes you choose one web framework over another? – The “classic” question for which there is probably no one all-encompassing answer.  Certainly there are lots of different factors to consider and it would be arrogant for me to try and suggest one.  Certainly JSF is a capable “general purpose” framework (even more so with JSF 2) but ultimately it comes down to what makes sense for YOUR project, with the skills of YOUR developers, and YOUR preferences.  One thing I will say is that even though there is a lot of this kind of comparison out there online, remember that JSF 2 solves many of the JSF problems so look at whether any discussion is comparing JSF 1.x and listing some of these issues that were fixed in JSF 2.
  • Which JSF extension, IceFaces, RichFaces, PrimeFaces, etc do you recommend? – Since all of these are JSF component libraries a lot of it depends on which has components that may be most useful for the type of application you are using.  All have rich Ajax support and IceFaces has an interesting “Ajax Push” model which lets you push data from the server to registered Ajax clients which can be useful for live updates if you actually need that in your application.  This also depends a little on if you are using JSF 2 or not as with the native Ajax support of JSF 2 these libraries may inter-operate better.  If you are using an earlier version of JSF you want to stick with only one for Ajax functionality.  PrimeFaces and IceFaces also have support for mobile applications which is certainly interesting given the growth of mobile development.
  • What is MyFaces? Is it another JSF implementation? – MyFaces is primarily a JSF implementation that is an alternative to the “standard” JSF implementation called ‘Mojarra’.  MyFaces also has a number of JSF component libraries that provide extensions to JSF.
  • What web framework is better, JSF 2 vs. Struts 2? –  I think if you say JSF 2 I would say JSF hands down.  We have Ajax integration with JSF built right in, rich ecosystem of JSF component libraries, much better options for managing state within an application, and those are just off the top of my head.
  • Any recommendations for resources to use when upgrading from JSF 1.x to JSF 2? – We are going to offer a “What’s new in JSF 2” training class.  In general I think what is different is you have different options for validating partial data sets or even changing the content of a form using the JSF 2 AJAX support which was much more difficult prior to this.  So I think the JSF 2 Ajax integration is where to focus a lot on.
  • Is JSF concept similar to .net aspx? – It is similar in that we are linking more directly the visual elements on the web page to variables and methods in objects on the server.
  • Are you aware of any companies/websites using JSF ??? – http://www.jsfcentral.com/trenches/ has some articles about people using JSF. Many of these are from when JSF was new but JSF 2 will make JSF more popular and not less.
  • Are people moving from struts 1.x to Struts 2 or are they moving more to JSF? – I think more people moved to Struts 2 when it came out but not as much now. Since Struts is kind of “dead” now I might look into moving to JSF since that will continue to evolve and offer new features for solving problems in the future.
  • How does JSF works with Web 2.0 (Dojo)? – It depends on why you are using Dojo.  If you use it just to get some Ajax functionality without as much JavaScript programming you might look at the native Ajax support in JSF 2.  If you specifically want some of the widgets available with Dojo you might look into DojoFaces which is an integration library for the two.

Java/JVM

I think another big source of questions had to do with the Java JVM, especially the 32-bit/64-bit software difference.

One thing that might help clear up a number of questions is to know that even though you want to install a JVM that matches the environment it is running in it doesn’t change Java itself.  The representation of Java primitive values is the same in all JVMs.  The only thing that changes is how much memory is potentially available to set the “maximum heap size” of the JVM and how the JVM represents memory addresses internally.  But since in Java you are not manipulating the memory directly anyway Java programs will not be affected.  Also, it is possible to run a 32-bit JVM on a 64-bit machine but not the other way around.  Typically the difference is not as important for development machines but would be important to match on the servers that may run Java EE applications where you should use 64-bit JVMs when available.

  • When your server has a specific OS, why do we even bother to run things on JVM? Isn’t JVM an extra layer not needed? – Certainly you lose a little something running in the JVM compared to “native” but with some of the performance improvements in recent JVM versions and even some of the research being done for “Real Time Java” this is less and less.  Besides, the portability you get is not just a “I don’t have to recompile my application for a different platform” portability it is a “I don’t have to rewrite every line of my application for a different platform” kind of portability which is more important when you think of the long life of typical applications.
  • What happens to the primitives in the language like int, will it be 32 bit int? or 64 bit if you declare them with just int? – The representation of primitives like ‘ints’ would be the same in both.
  • So JVM is hardware platform specific? You need to download the right JVM for your platform? – Yes, you always want to match as closely as possible.
  • When an application is run and enters main method, how is JVM handling processes? Do we still get separate PID for each program? – Every time you run the application a separate JVM process will be created by the machine to run the program.  If a program is “multi-threaded” it would run several threads of program computation within the same process.
  • So applets do not run on top of JVM? – Applets do run in a JVM but one that is installed as a plugin to the browser the Applet is running in.
  • Difference between JDK, JRE and JVM? – JVM runs the program. JDK has the development tools to compile apps, JRE just runs stuff.
  • If I develop a program on Java EE 6 or JDK 1.6 can I run it on Java EE 5 or JDK 5/1.4 or lower? – In general no.  Part of it is the code will be compiled against the more recent version and not recognized on the older JVM and the other part because if you use parts of the newer API those will not be available in the older JVM.  The reverse is true though, if you compile or design a program for an “older” version of Java you can run it in a “newer” environment.
  • JVM 64 B program will it run in 32B? – Technically there are no “64 bit programs”, they are just Java programs and will run the same in both environments.
  • Does OS makes difference Unix/ Windows? – Yes in terms of JVM you need to install, no in terms of how you write your program.  One of the portability benefits of Java is you can just write the program against the generic “Java Virtual Machine” and they will operate correctly on different platforms.  One example is working with files where the concept of “open the file, read the file, write to the file, save the file” would be the same.  The only difference might be how you specify the file name but if that is passed in as a program argument there will be nothing different in your code.

Spring

In one poll question there were a high number of people who said they did not use Spring so I’m not surprised this was another area for questions.

  • When to prefer Spring WebFlow, in comparison to Spring MVC? – Spring WebFlow is definitely for when the application has a fairly distinct navigation between pages in the application so this is modeled in a “flow”.  This would be the “Show cart, collect shipping address, billing address, enter payment details, confirm order, display confirmation” type application.  Spring MVC is used for when all of the action needed is handled by a single request that collects all the data needed.  This could be a “list appointments, add new appointment, cancel appointment, etc” application where each action is pretty self-contained.
  • I still do not see, how spring replaced EJB, both are very different technologies. – Both are very different in implementation but similar in some of the benefits you can get by using them. Also, Spring can be used outside of a Java EE server.
  • What are the other features that add value in Spring? – Spring has utility modules for JMS, JDBC (if you don’t use JPA), etc. It also has a few web frameworks that are specific to Spring. Spring is also good if you want a full dependency injection model but can’t migrate to a Java EE 6 server.
  • Tomcat is very popular so people must be using Spring or similar to get EE function like transactions and queuing? – This is true and one of the things Spring tries to “sell” people on.  The reality though is now that there are lots of open source Java EE application servers perhaps the model of “Tomcat is so simple” has outlived the reality.  Now if you want to add any kind of Java EE functionality to Tomcat you spend a lot of time doing this when using a Java EE certified server would give you more functionality “out of the box” and you would need to rely less on Spring or doing it on your own.  Spring originated when you had to pay for Java EE (J2EE) functionality which was some of the original motivations for creating Spring.

Eclipse

Eclipse was the most popular development tool of people attending the session so some questions about that also.

  • Can you comment on Eclipse. does it have a highly productive drag and drop tool? – Eclipse does have great drag and drop for JSF. When you use the “Web” perspective with the “Palette” view and open a JSF page with the “Web Page Editor” (not the JSP editor).
  • Are the Eclipse plug-ins downward compatible? – I think this is asking if a plugin is compatible with one version of Eclipse would it be compatible with previous versions and generally the answer is no.  The good thing is now Eclipse has a very good way to add new software from “update sites” and these can check your version of Eclipse and make sure to install the correct version.  Eclipse has also gone to “naming” the yearly releases and this is the place to start when looking for compatible Eclipse plugins and environments.  If there is a specific plugin you want to use make sure to use an Eclipse version that is supported.
  • What tools would you suggest for a brand new Java web application? – Since there are free Eclipse plugins for every major server now this is definitely the place to start.  The place to start would be the ‘Eclipse IDE for Java EE Developers’ from the Eclipse downloads section.  Make sure you get a version that is compatible with the plugin that would allow you to work with your server and you may need to start with a slightly older Eclipse version.
  • What is a recommended version management Eclipse plug-in? – Assuming your version control system is set you would get an Eclipse plugin to match this.  Often they are provided by the source of the version control system.  If your version control system is not set evaluate based on the features of version control systems as this will affect things more.  Certainly you can use “availability of Eclipse integration” as a criteria but generally all provide some kind of plugin.  Also remember that the Eclipse plugin is not the version control system itself but just how you can work from within Eclipse and perform actions in the separate version control system.
  • Does regular Eclipse have a WebSphere plugin? – IBM just released a FREE Eclipse plugin that provides server management tools for WebSphere 7.0 and 8.0.  This is so groundbreaking I wrote a separate post about it.

Misc. Questions

  • What is POJO? – POJO stands for “Plain Old Java Object” and means generally that a Java component does not need to extend another class or implement an interface of some technology.  In general POJO has been used to indicate “simple” Java components that may at most have an annotation enabling certain features although some might argue that even the presence of an annotation links it to a specific technology or framework.
  • Java SE vs Java EE – which one has more API? – Since Java EE extends Java SE it includes everything from Java SE and therefore is always more.  In terms of comparing Java SE to (Java EE – Java SE) I’m not sure.  Probably depends on version somewhat.
  • Hibernate struggles with stored procedures in the RDBMS. Is JPA any better? – JPA does not add anything specific to stored procedures.  The closest is running a “native query” where you are supplying the SQL directly which could then be a stored procedure.  The support for this is pretty much going to be up to the JPA implementation which would be different by default on different servers.
  • Can EJBs be nested? – Yes, you can have one EJB invoke methods on another EJB

Resources

A few people had questions about good books/resources.

  • Can you recommend a good book for Spring 3 MVC? – I don’t have a specific one but make sure you get one that covers the exact version of Spring you will use.  Spring MVC is part of the main Spring project and is not it’s own project.  Spring MVC also changed quite a bit in Spring 3.0 which is why it is so important to match versions.
  • Can you recommend a good book for JSF 2? – I have started looking at ‘Core JavaServer Faces’ 3rd edition which is updated for JSF 2.  This book has always had good reviews and what I like about the new edition is it clearly identifies what is new to JSF 2 so those that are experienced can get to the new stuff while those new to JSF get value from the whole thing.  This is also one of the ONLY Java EE books (on any part of Java EE) that even mentions the JSR 299 – Contexts and Dependency Injection (CDI) standard and how it integrates with JSF.  CDI is a major new technology and I’m shocked how it is really not covered practically at all in the current Java EE 6 books available so this was a big selling point for me.
  • Can you recommend any good resources for going from java 1.4 to java 1.6? – I don’t have any specifically but if you are talking the difference between Java SE 1.4 and Java SE 6 (1.6) the main differences were actually introduced in Java SE 5.  This introduced things like generics, enhanced for loop, etc and many more things that affect the language itself.  Our Advanced Java class has always had a chapter on major changes in the last few Java versions and the differences for Java SE 5 are more than those for Java SE 6.  Now if you are talking about differences between J2EE 1.4 and Java EE 6 that is a different story as there are many more changes from Java EE 5 to 6 in my opinion.

Poll Questions

I think it would be good to also recap the results of the poll questions for some idea of what tools and technologies people were using.  Although most poll questions only got a 40-50% response rate I would assume they were fairly representative.

Question – Which IDEs do you use?

  • Rational Application Developer (RAD) – 40%
  • “Regular” Eclipse (free) – 66%
  • MyEclipse – 11%
  • SpringSource Tool Suite – 3%
  • NetBeans – 10%
  • JBoss Developer Studio – 15%*
  • IntelliJ – 12%*
  • JBuilder – 31%*

* – These responses were part of a second question I created which had a much lower response rate.  These responses may have been elevated by people who answered the first question and didn’t use any additional tools not answering this question.

Analysis – This confirms what I see with our clients which is that RAD is still popular because most WebSphere shops have had to use it.  The release of the FREE Eclipse plugins for WebSphere 7.0 & 8.0 may change this.  The “Regular” Eclipse is the most popular although there is some usage of just about all of the other major tools available.

Question – Which Java EE Technologies are used by your applications?

  • EJB 3.x (Java EE 5/6) – 52%
  • EJB 2.x (J2EE 1.3/1.4) – 30%
  • JavaServer Faces (JSF) – 35%
  • JAX-WS web services – 49%
  • Java Persistence (JPA, Java EE 5/6) – 27%

Analysis – I was pleasantly surprised to see the EJB 3.0 adoption so high.  Given that, however the JPA rate seemed a little low.  Maybe migrating to JPA, which is a completely new technology, is different than simply programming existing EJBs a different way.

Question – Which web framework is used for NEW applications?

  • Struts – 7%
  • JSF – 23%
  • Spring MVC – 20%
  • Servlet/JSP (without anything else) – 11%
  • “Other” – 19%

Analysis – Obviously for this one what constitutes the “Other” response would be reason to open up a new skirmish in the “framework wars”.  My main goal though was to see how some of the new frameworks like JSF and Spring MVC held up against the “old” choices of Struts and Servlet/JSP.  It is clear that JSF is the new “standard” although Spring MVC had more responses than I expected.  In my opinion too many people are still using Servlet/JSP since that just forces you to do too much work or maintain your own custom “framework” anyway.  Based on how many people weren’t really aware of JSF when we discussed that perhaps that will change.

Question – Which Spring version do you use?

  • Don’t use Spring – 63%
  • Spring 3.x – 16%
  • Spring 2.5.x – 14%
  • Spring 2.0.x – 11%

Analysis – I was surprised to see so many people not using Spring at all.  Of those that did use it however the biggest percent was on the most recent Spring 3.x version proving that it is easier to move to a new Spring version than a new Java EE/server version.

Question – How soon until you move to a server that supports Java EE 6?

  • We are using one now – 14%
  • Within 0-3 months – 2%
  • Within 3-6 months – 4%
  • Within 6-12 months – 22%
  • More than a year from now – 59%

Analysis – This confirms that using a new server version is not something that happens overnight although a surprising number are already using one.

No Comments

Uploading Files Using XMLHttpRequest Level 2

As a part of HTML5, XMLHttpRequest API is undergoing enhancements. You can now upload files to a server and display a progress bar. Files are uploaded in their raw source format. That is not same as uploading a file using the old <input type="file"/> approach.

The basic steps are as follows:

  • Use the HTML5 File API to let the user select a list of files or drag and drop files on an element.
  • Upload these files using XMLHttpRequest Level 2.

Setting Up a Basic UI

Here is a very simple UI for dropping files and uploading them.

<div id="fileDrop">
       <p>Drop files here</p>
</div>
<a id="upload" href="#">Upload files</a>

You can style it better in real life. We will just show the basic code here.

Setting up File Drag and Drop

Within a <script> element, add these lines.

var files;

//Stop default actions
function stopProp(ev) {
    ev.stopPropagation();
    ev.preventDefault();
}

function init() {
    var dropZone = document.getElementById("fileDrop");
    //dropZone.addEventListener("dragenter",  stopProp, false);
    //dropZone.addEventListener("dragleave",  stopProp, false);
    dropZone.addEventListener("dragover",  stopProp, false);
    dropZone.addEventListener("drop",  setFile, false);
	
    var upload = document.getElementById("upload");
    upload.addEventListener("click",  doUpload, false);
}

//Save the dropped files
function setFile(e) {
    e.stopPropagation();
    e.preventDefault();

    files = e.dataTransfer.files;
	
    alert(files[0].name);

    return false;
}

window.onload = init;

function doUpload(e) {
  //Upload code here
}

Basically, when files are dropped, we save them in the files global variable. Also, we set the onclick handler of the Upload link to the doUpload() method.

Upload Files

To keep things simple in this article, we will upload only one of the dropped files. You can easily change the code to upload all files.

Fill out the doUpload() method as follows.

function doUpload(e) {
    var xhr = new XMLHttpRequest();
    var file = files[0];
    xhr.open("POST", "upload.jsp");
    //Set a few headers so we know the file name in the server
    xhr.setRequestHeader("Cache-Control", "no-cache");
    xhr.setRequestHeader("X-Requested-With", "XMLHttpRequest");
    xhr.setRequestHeader("X-File-Name", file.name);

    //Initiate upload
    xhr.send(file);

    stopProp(e);
}

The Server Side

The browser sends the contents of the file in the request body. A typical request will look like this.

POST /UploadTest/upload.jsp HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 48
Origin: http://localhost:8266
X-Requested-With: XMLHttpRequest
Cache-Control: no-cache
X-File-Name: Story.txt
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7
Accept: */*
Referer: http://localhost:8266/UploadTest/index.html
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: JSESSIONID=0CDE28BDE75AD2B9486070FDA5AED6DB

Long long ago....

In a galaxy far far away...

If the file is binary, all the bytes will be sent in raw form without any kind of base 64 encoding.

To read this type of data, you will need to get the input stream of the request. In a Servlet or JSP, this can be done by calling ServletInputStream getInputStream() method of HttpServletRequest object. Using the input stream, you can read all the data.

1 Comment