Category: Uncategorized

HR Web Application – Tomcat & UCP

Many years ago a sample web application for using JDBC was published as part of the oracle-db-examples GitHub repository. The HR Web Application example was the starting point for some to build their first simple web interface to their database. Who knows how many simple, in-house applications have this as their inspiration?

The example had a particular focus on Apache Tomcat and the steps to getting it built and deployed where simple for the standard Apache Tomcat setup at the time.

-- Get the code
git clone https://github.com/oracle/oracle-db-examples.git
cd oracle-db-examples/java/HRWebApp
-- Copy the tomcat-users.xml and start tomcat
cp tomcat-users.xml $CATALINA_HOME/conf
catalina.sh start
-- Build the war file and deploy it
mvn package
cp target/JdbcWebSamples.war $CATALINA_HOME/webapps

By copying the JdbcWebSamples.war into the $CATALINA_HOME/webapps directory, the web application is automatically deployed by Apache Tomcat. The context path is based on the file name so the URL is http://localhost:8080/JdbcWebSamples/

Use your browser to access the web application.

The tomcat-users.xml defined two new users: hradmin and hrstaff. Both have welcome as their password. Login and click on the List All menu item to see the list of HR.EMPLOYEES records. Of course, that’s if the database connection details are correct.

Your JdbcBeanImpl.java will have to be changed to point to the correct database in the getConnection() method.

 public static Connection getConnection() throws SQLException {
    DriverManager.registerDriver(new oracle.jdbc.OracleDriver());
    Connection connection = DriverManager.getConnection(
         "jdbc:oracle:thin:@//mydatabaseserver:1521/orclpdb1", 
         "hr", 
         "hr");

    return connection;
  }

Make the above change for your database, run mvn package and copy the JdbcWebSamples.war to $CATALINA_HOME/webapps to get that running with the new connection details.

Take a look through JdbcBeanImpl methods in detail and you’ll notice that getConnection() method is called using the try-with-resource syntax so the connection is automatically closed after every operation. It’s a good practice because it does not leave INACTIVE sessions on the database. In fact, if you check your database v$session you shouldn’t find any records.

SELECT
    *
FROM
    v$session
WHERE
    program = 'JDBC Thin Client'
    AND schemaname = 'HR';

Creating a database connection for every request does have an overhead and if that is a remote database the network latency could be sufficient to cause problems for the users of the web application. To see the impact of creating a connection every time, let’s add some basic logging on the elapsed time to that getConnection() method.

 public static Connection getConnection() throws SQLException {
    final long start = System.currentTimeMillis();
    DriverManager.registerDriver(new oracle.jdbc.OracleDriver());
    Connection connection = DriverManager.getConnection(
         "jdbc:oracle:thin:@//mydatabaseserver:1521/orclpdb1", 
         "hr", 
         "hr");
    final long end = System.currentTimeMillis();
    logger.log(Level.INFO, 
         "Creating a connection duration(ms): " + (end - start));

    return connection;
  }

Like before, make the above change for your database, run mvn package and copy the JdbcWebSamples.war to $CATALINA_HOME/webapps. Every time the list of employees is retrieved, or any action that involves the connection, the cataline.out will show a log message.

Creating a connection to my remote database takes over 2 seconds!

One way to address this is by using Oracle’s Universal Connection Pool. This could be used through a data source defined in the Tomcat configuration but it can also be done used programmatically. Let’s do that.

The first thing is to upgrade to a more recent version of JDBC/UCP. Not the latest 21.1 version, more on that later, but we’ll use 19.9.0.0 for now. Note that the group id has changed from com.oracle.jdbc to com.oracle.database.jdbc and that’s what we’ll use in the pom.xml

Change this

    <dependency>
      <groupId>com.oracle.jdbc</groupId>
      <artifactId>ojdbc8</artifactId>
      <version>12.2.0.1</version>
    </dependency>

to this

    <dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ojdbc8</artifactId>
      <version>19.9.0.0</version>
    </dependency>
    <dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ucp</artifactId>
      <version>19.9.0.0</version>
    </dependency>

Now add a class to create and configure a Universal Connection Pool PoolDataSource. Let’s call it JdbcSource in the com.oracle.jdbc.samples.bean package.

package com.oracle.jdbc.samples.bean;

import java.sql.Connection;
import java.sql.SQLException;

import java.util.logging.Level;
import java.util.logging.Logger;

import oracle.ucp.jdbc.PoolDataSource;
import oracle.ucp.jdbc.PoolDataSourceFactory;

public class JdbcSource {
  JdbcSource() {

    //Create pool-enabled data source instance.
    this.pds = PoolDataSourceFactory.getPoolDataSource();

    //set the connection properties on the data source.
    try {
      pds.setConnectionPoolName(POOL_NAME);
      pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
      pds.setURL("jdbc:oracle:thin:@//mydatabaseserver:1521/orclpdb1");
      pds.setUser("hr");
      pds.setPassword("hr");

      //Override any pool properties.
      pds.setInitialPoolSize(2);

    } catch (SQLException ex) {
      logger.log(Level.SEVERE, null, ex);
      ex.printStackTrace();
    }
  }

  public Connection connection() throws SQLException {
      return this.pds.getConnection();
  }

  private final PoolDataSource pds;
  private static final String POOL_NAME = "JdbcWebSamples_pool";

  // Singleton data source. Not a great pattern but simple for demonstrations.
  public static JdbcSource INSTANCE = new JdbcSource();
  static final Logger logger = Logger.getLogger("com.oracle.jdbc.samples.bean.JdbcSource");
}

The above will create a pool of database connections that can be reused every time connection() method is called. The pool is initialised with 2 connections. That means there will be two sessions on the database that will be INACTIVE most of the time. The web application may deal with concurrent requests so having an extra connection ready will help with that additional load.

Let’s revisit that JdbcBeanImpl getConnection() method and change it to use the new pooled connection. It’s not all that complicated…

 public static Connection getConnection() throws SQLException {
    final long start = System.currentTimeMillis();
    DriverManager.registerDriver(new oracle.jdbc.OracleDriver());
    Connection connection = JdbcSource.INSTANCE.connection();
    final long end = System.currentTimeMillis();
    logger.log(Level.INFO, 
         "Creating a connection duration(ms): " + (end - start));

    return connection;
  }

Like before, make the above for your code, run mvn package and copy the JdbcWebSamples.war to $CATALINA_HOME/webapps. Everytime the list of employees is retrieved, or any action that involves the connection, the cataline.out will show a log message for the first connection taking a long time, but every subsequent call takes milliseconds.

There’s an overhead for the 1st connection but subsequent requests get a connection in milliseconds

The query on v$sessions will show 2 INACTIVE sessions at least. As more concurrent requests are received, the pool size will grow automatically so more sessions could be created. In fact, hit that List All menu item repeatedly 20 or 30 times and you’ll see the number of v$sessions for HR schema grow. There’s more to explore here on setting UCP properties for optimising pool behaviour.

Upgrade to JDBC / UCP 21.1

Let’s leave that as a bonus point exercise for you and discuss upgrading to 21.1.0.0. This version has support for defining data sources in JBoss and Spring. That enhancement does break these example web applications which include the UCP jars in them.

Update the pom.xml to use the new 21.1.0.0 JDBC and UCP jars. 

    <dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ojdbc8</artifactId>
      <version>21.1.0.0</version>
    </dependency>
    <dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ucp</artifactId>
      <version>21.1.0.0</version>
    </dependency>

Rebuild and deploy to see the following SEVERE error messages.

SEVERE [Catalina-utility-1] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file
SEVERE [Catalina-utility-1] org.apache.catalina.core.StandardContext.startInternal Context [/JdbcWebSamples] startup failed due to previous errors

The localhost log will have more details.

SEVERE [Catalina-utility-1] org.apache.catalina.core.StandardContext.listenerStart Error configuring application listener of class [oracle.ucp.jdbc.UCPServletContextListener]
	java.lang.NoSuchMethodException: oracle.ucp.jdbc.UCPServletContextListener.<init>()
		at java.base/java.lang.Class.getConstructor0(Class.java:3427)
		at java.base/java.lang.Class.getConstructor(Class.java:2165)
		at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:151)
		at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4607)
		at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5146)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717)
		at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690)
		at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
		at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:978)
		at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1849)
		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
		at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
		at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
		at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:773)
		at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:427)
		at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1620)
		at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:305)
		at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
		at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1151)
		at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1353)
		at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357)
		at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1335)
		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
		at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
		at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
		at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
		at java.base/java.lang.Thread.run(Thread.java:832)
SEVERE [Catalina-utility-1] org.apache.catalina.core.StandardContext.listenerStart Skipped installing application listeners due to previous error(s)

Put simply there is an annotated class that Tomcat discovers in the web application classpath and tries to initialise it, and it fails. It’s not a great work around, but a quick fix is to use the web.xml to tell Tomcat the names of components it should attempt to initialise. In this example web application there are two servlets defined: GetRole and WebController. These are in the com.oracle.jdbc.samples.web package. We’ll mention their names explicitly in the web.xml. Add this absolute-ordering entry just after the login-config.

  </login-config>
  <absolute-ordering>
    <name>GetRole</name>
    <name>WebController</name>
  </absolute-ordering>
</web-app>

It may not be a feasible work around for all, and it sort of defeats the purpose of having annotated servlets, but explicitly mentioning their names in absolute-ordering makes upgrading the JDBC/UCP jars possible.

When machines learn to do what you do

For a brief time, I was a spy! On a clandestine mission, sending and receiving encrypted messages with my associates. That was for about 5 minutes and then I had to give someone else a go.

Exploring the city of spies @spionagemuseum

On a recent trip to Berlin I visited the German Spy Museum and spent hours upon hours enjoying the exhibits, the interactive displays, and learning so much about the history, and ingenuity of spy craft. One of the many interesting topics was the history of industrial espionage. From navigating to new worlds, metallurgy secrets and weaving cloth, the wealth to be gained was immense. There were clear advantages in these innovations that those who had them went to great lengths to protect what they had. Some others went to great lengths get a hold of it too.

One such innovation, was the automated weaving loom.

The history around the automation of the weaving loom was something I was already familiar with, particularly as Jacquard’s loom inspired Charles Babbage in early computational device design. The Jacquard Loom automated the work of weavers. Changing the punch cards changed the pattern, giving the weaver endless ways to “program” this device and to create intricate tapestries, damasks, brocades and other fabrics. What I did not know before writing this article was the difference between damask and brocade fabrics. Which goes to show that you can learn knew things. More on that later.

What’s the big deal about this automated loom then? Well, traditional silk weavers could produce approximately 2cm of complex fabric in a day. The skilled Jacquard Loom operator, however, could create approximately 50cm of fabric in the same amount of time.

The automated loom, which allowed for an incredible rise in cloth production, and more complex designs, was over 100 years in the making though. The challenges were not just technical. There was considerable opposition to machines destroying the livelihoods of weavers and those that worked in the weaving industry.

No doubt there were some who saw the attempts at automation in the mid 18th century and thought there was no way these clumsy, unreliable, and high maintenance contraptions would ever take the place of a human. Similar to how the Roomba of today does not put any cleaner in fear of replacement. Yet.

The topic of automation eliminating jobs is not a new one. For a many years some researchers and analysts have referred to the industrial revolution as the first machine age where automation changed the nature of repetitive manual work. Some consider us in a second machine age where it is not simply manual work being replaced. Research shows that there are worries about job automation globally.

A quick side note…the Washington Post review of The Second Machine Age, a book by two MIT professors who have coined the phrase, is exceptionally good. Much better than what I could have done, so I’ll link to that Washington Post review.

Back to the machines…What makes the industrial revolution significant over other periods of innovation is that before the first machine age, technological innovation resulted in larger populations, not wealthier populations. Take a look at those Our World In Data charts: https://ourworldindata.org/economic-growth

The real difference was on scale. Machines could make something that was prohibitively expensive, or time consuming, possible. A modern example would be the use of artificial intelligence to analyse sales calls which is what sales managers would do as part of developing their team. The good news is that one manager could potentially have a larger team. The bad news is that there may not be a need for so many managers.

Life long learning

A generation ago, the half-life of a skill was about 26 years, and that was the model for a career. Today, it’s four and half years and dropping

Indranil Roy – Future of Work

Looking back again at the weavers, those who continued in the industry became designers, putting an emphasis on the design and planning skills that were already apart of the weaver job description. They would also have learned the mechanics of the machines that was transforming their industry forever. This is a useful guide for how we should plan for more automation in our careers: constantly updating skills.

Which skills are worth updating though? Perhaps more importantly, how can one identify the skills that one has an ability for? Joseph E. Aoun suggests that there are three disciplines to consider: technical, data and human. These are effectively…

  • awareness and familiarity with the automation technologies emerging.
  • cognitive skills and data literacy so that more information-centric skills can be applied to understand the world around us and devise new strategies based on that understanding.
  • an understanding of creative practices and cultural experiences that develops the ability to take information from one context and apply it to another.

The technical and data abilities are perhaps the easiest to justify spending time and money on because the connection between them and work is much clearer. Deloitte’s Future of Work Centre of Excellence identifies these talents that are needed in the work place. These talents can help assess, and take advantage of, a process or level of quality that previously was too expensive, or impossible, until now. An employee dedicates a few hours a week to study digital marketing and before you know it, the business has a social media strategy.

However, the soft skills that are going to pay off are harder to identify, simply because of the human element. A passion for travel, baking, music or sports may not lead to a new career but can be very rewarding and by sharing that, may reveal new connections or opportunities. That digital marketing expert mentioned earlier may only have taken the initiative because of their interest in photography.

One of my favorite podcasts is NPR’s How I build this and here are just a few examples of people taking their interests, seeing that there’s a gap in the market, and doing more: SoulCycle, KickStarter, method, Drybar. The stories are inspirational but make it clear that even with planning, collaboration and support, success did not come easy.

You may be fortunate to have an employer that supports skill development, either by providing leave to study, or even a contribution to the cost of studying. Quite often the effort that goes into making the case to your employer for training is exactly the right measuring stick to gauge if this is something you really want to do. Perhaps you are not so fortunate and will have to cover the time and cost out of your own free time and pocket. If that’s the case, visit your local library or enterprise board because there may be grants or other schemes available to lessen the burden. Many OECD countries are coming around to be more proactive in supporting skills development in the workforce.

In summary, technological advances will make certain fields of expertise obsolete but will introduce new ones that we should prepare for. The best way to prepare is through learning new skills, but we do have to identify and choose the right ones. There are no easy answers to making that decision. That is, until someone comes up with an AI for exactly that purpose.

Tailoring Fusion CRM

The journey for the Fusion CRM development team has been a long one. What a great feeling for all of us when Fusion Applications was officially released this year as Generally Available. During his keynote speech at this year’s Oracle OpenWorld Steve Miranda reiterates that statement. Also during his session there were some great demonstrations of Fusion CRM. Anthony Lye demonstrated (about 25 minutes into the presentation) how the application can be extended at run time. Design time at run time features are made possible by a number of Fusion Middleware technologies including WebCenter and MDS (pdf 592kb). Extending the application, as Anthony Lye demonstrated, is one form of tailoring that can be performed with the out of the box application. Other forms of tailoring include personalisation and customisation (personalization and customization in the documentation). Both concepts are very similar in that changes to the application are persisted via MDS but they are differentiated by the scope of the change and for whom the change is made.

Available to all users.

Put simply, personalisation is a tailoring action performed by an end user for themselves to set their own preferences for how they work with the application. These changes can be to include dynamic content such as Business Intelligence reports or a syndication feed from an external source. The key point is that these are personal preferences explicitly specified by the user for the user by editing the page via the WebCenter Page Composer. A user can also perform implicit personalisation of the Fusion CRM application without ever touching Page Composer simply by hiding columns in a table, or rearranging their order relative to each other by dragging and dropping. These personal preferences are also reflected in the shared components across Fusion CRM including the Activities UI components for Notes, Interactions, Appointments and Tasks. Setting a preference for the display width of the comments column for Tasks in the Marketing application is reflected in Opportunity Management.

Only available to users with Administration privileges.

In the context of Fusion CRM, customisation is a modification made by an administrator, or someone with the appropriate permissions, to the application for a set of users. This set, referred to as Customization Level, can be defined as all users for the installed application, internal users, external users or users with a particular set of roles. For the Partner Relationship Management functionality in Fusion CRM V1 the internal vs external layer was introduced so that a Channel Administrator could tailor how they wanted their partners (external users) to see the Fusion CRM screens. Both internal and external users are interacting with the same screen, but the external users may see a lot more branding and content more suitable for partners rather than employees.

From a development perspective, MDS terminology can make things a bit confusing because irrespective of the nature of change, MDS refers to it as a customisation. There are many cross platform components used in Fusion CRM that are individual products in themselves. Getting to grips with the terminology used by these different product teams and understanding the context has been a challenge, but really rewarding when you see the results.