Maximum Security Architecture

In one of the past posts, I have just listed different technologies that are available in the Data Security area. (in Feb 2008)

With Oracle 11g database, the security focus has taken more methodical and architectural approach.

To put things together data security is placed under the following broad (four) categories:
1. User Management
2. Access Control
3. Encryption and Masking
4. Auditing/Monitoring

Just like Maximum Availability Architecture for Highly available architectural patterns, we can call this as Maximum Security Architecture for highly secure architecture....

One should choose the required options and implement it properly to really make the data SECURE!

This Link gives more details of MSA on Oracle 11g database.

Enterprise Data Fabric - data grids

Enterprise Data Fabric or "in memory data grid" can improve the distributed or clustered application performance dramatically.

The Problem:
Distributed applications need to share the "application objects" across multiple processing nodes. As the application objects are not "relational" there is a need for the Object Relational Mapping (ORM) to share the application objects using a relational database.

The ORM technology involves converting the object into a set of relational (table/column) information and and turns slow.

The Solution:
Using an in-memory data grid to store the application objects on distributed multiple node cache and managing the transactions in once and only once manner.

This will dramatically improve the performance and scalability of the application.

This is also going to be a key enabler of the Private PaaS Cloud Computing in future.

IBM WebSphere eXtreme Scale and Oracle Coherence are the examples of Distributed Caching Platforms that provide Enterprise Data Fabrics or in-memory data grid solutions.

Let us see how this technology shapes up the future!

migration, upgrade, enhancement

Migration of a software application from one platform (32bit OS1 on xxx hardware) to another platform (64bit OS2 on yyy hardware)

Upgrade of a software application involves upgrading the software from a lower version to a higher version (like XX version of database to YY version of database)

Both the above activities are technical and no business benefit is directly attributed to this type of activity on the software applications.

An application enhancement will involve providing a new feature or functionality to the end-user of the application. Sometimes when the application is majorly enhanced, the new enhanced application can get migrated to a new platform and/or the underlying technology stack upgraded to higher version.

The key question I am dealing in here is:
When there is no "enhancement" involved, purely a migration and/or upgrade is planned:
a. is it good to do both together in a big bang? OR
b. is it good to migrate first to the new platform and upgrade the software later? OR
c. is is good to upgrade the software first and then migrate to new platform?

a. One can do it together but if there is an issue it is difficult to find the cause of the problem due to new platform or new software version.... each can point fingers at the other....
b. is a good choice if the platform migration is more complex than software upgrade.
c. is a good choice if the software is more stable on the higher version and if any specific platform related issues are fixed in the higher version.

If the software application has multiple tiers with database, application server etc., one should carefully choose the overall path of migrate and upgrade. Making sure the impact on the business SLAs is minimized; It is also important to have a very good "back-out" plan, if something badly goes wrong!!

Performance Analysis & Tuning

Performance of any system is defined as "accomplishment of a 'task' measured against agreed quality, cost and time."

The software tasks are primarily divided into "transactional" tasks and "analytical" tasks.

One big area in Software Systems management is the Performance Analysis and Tuning.

How to measure the performance?
The basic thing needed for "Performance" is the "Measurement" - What is to be measured and How?
Generally people measure the "response time" (time taken to complete the task) if the task is transactional
and
throughput (number of tasks performed in a fixed unit of time) if the task is analytical.

So, in both the cases what is measured is "time". What happened to the original definition that consisted the "quality" and "cost"?

In Software systems the "tasks" are automated and once the automation is tested well, "quality" is guaranteed for the task.
The "cost" of performing the automated task of a software application is the computing resources (CPU and memory) that are allocated to the application and how well the application can use additional resources to carryout the tasks.

Assuming the application can take full advantage of available CPU and Memory resources allocated to it, "TIME" becomes the best measure to analyze the performance of a software system.

How to Tune the system?
Reduce the amount of time taken to perform the 'task'. As simple as that.

Is it that simple?
Let us take a close look at this approach. Can we really reduce the time taken to perform a task?
Answer is "NO". If a task takes "x" amount of seconds to do, it takes "x" seconds to finish it.
But,
Every Software task has two components of this "time". What are they?
a. The time taken to actually DO the work
b. The time taken while waiting for a dependent sub-task to get completed.

So, Now if we ask the same question Can we reduce the time taken to perform a task?
The answer is YES - by reducing the time "waiting"!

Performance Tuning is all about scheduling and executing the sub-tasks of a "task" optimally such that the waiting time is minimized (if not completely removed) thereby using the available resources optimally and reducing the "cost" of performing the "task" That is the secret of performance tuning!

Platform As A Service - Private Cloud

Sometime back, Oracle has published this white paper.

This gives a clear vision of where Oracle is heading in the technological direction of Cloud Computing.

By standardization on the technology stack for enterprise application and fusion middleware enhancements, the PaaS seems the natural direction for enterprises in medium term.

Link to my past blog post on Private Clouds Here.

SOA Governance standalone? or merge with IT Management?

"Governance" in general and SOA Governance in particular has been a buzz word for sometime now.

My definition for Governance is making sure that a "thing" works as it is supposed to work. Replace the "thing" with SOA, IT, state, country etc.,

One of the emerging trends for past few years is "Business Transaction Management" for managing the composite IT applications configuration, performance and "Governance"

Traditionally the IT management tools and frameworks have looked at the IT as operations management, quality management, performance management, configuration management etc., disciplines.

In this trend the problem of IT management is from a business point of view. That is good. So, what is happenning in the technology?

1. Application Performance Management
2. Composite Application Management using run time discovery of relationships between the components (services) of the application
3. Business transaction management tools

Here is the question:

Will the new trend merge in traditional IT Management tools?
or
A standalone set of new SOA Governance tools emerge due to the new trend?

Based on my "UNIX" philosophy, I think an integrated set of tools that consist of traditional IT management with the new SOA Governance would be a best fit to tailor to the needs of contemporary corporates!

Let us wait and see....

Hybrid Columnar Compression (HCC)

Oracle has introduced a new data storage technology in Exadata V2 called Hybrid Columnar Compression. This method uses a "compression unit" that is made of several rows; it is physically stored in compressed "column vectors" in multiple blocks of storage.

There are two flavors of columnar compression that are optimized for "QUERY" (warehouse compression better for executing queries) and "ARCHIVE" (best data storage for archive compression)

Each of the QUERY and ARCHIVE come with another modifier to set the "LOW" or "HIGH" compression mode.

This technical white paper gives an overview of the new technology.

This article on Oracle Magazine may also be useful in understanding this new feature.

More data can be stored in "less" space. Good!

sizing a data center management solution

Introduction:

A typical data center management solution will have a "data store" for storing
a. configuration data
b. monitoring data

On regular intervals it will collect the data and upload it to the central data store.

A host, database, listener, application server instance etc., are the managed entities that will need monitoring and management.

Before implementing the solution, every organization will try to "size" the infrastructure requirements for the solution.

The sizing involves estimating the resource consumption for the solution.
1. disk storage
2. memory
3. cpu cycles required
4. network bandwidth requirements

The Problem:
The complexity involved in sizing a data center management solution is mainly due to lack of clarity of definition.
for e.g., customer want to monitor 10 databases on 10 servers. (one DB on each server)
Depending on the database version and weather it uses ASM for storage or it is on a cold failover cluster and several other considerations the number of managed entities will vary. A database can have only one tablespace and a single datafile or a database can have 10K tablespaces with 100K datafiles. If customer want to monitor the database space usage by tablespace a database with 10K tablespaces will produce 10K times more data when compared to a database with 1 tablespace.

Each metric (the monitored datapoint) can also be collected once in every 5 minutes or once in an hour. The collection frequency may vary based on customer requirements.

A simple formula is number of metrics ( number is at lowest granule) * number of collections per day gives the total number of metric values per managed entity. Multiplying this number with bytes required to store an average metric value gives the bytes required to store the metric values per managed entity.

If customer wants to keep this data at this granularity for a week, then the storage requirement is number of days of retention at raw granularity * bytes required to store one day data. This multiplied by the number of managed entities gives the total storage requirement for this type of managed entitiy (e.g., database)

The same exercise need to be repeated for all types of managed entities to get the storage requirement.

Collecting that many bytes over a day means transferring that data over the network from the host where the managed entity resides to the host where the management data store resides. That gives the network bandwidth requirement.

This data need to be rolled up to a hourly average and daily average for keeping it for historical trending. One need to calculate the space requirements and processing requirements in this rolling up process.

The old data need to be purged away from the data store. It needs processing cycles.

All the managed entities should also have a set of configuration data. That need to be collected, compared on a regular basis and kept up-to-date. One need to compute the resources required to collect, compare and store the original and changed values of different configurations as they evolve.

Managing all the "heartbeats" between the central management solution and the monitored hosts requires a proactive two way mechanism. This needs processing capacity.

Monitoring the websites with synthetic transactions by periodically playing a pre-recorded transaction to make sure the application works require additional set of processing, storage, memory etc., This need to be added to the above estimate.

A provisioning solution that is used to deploy new instances of databases, application servers will need the GOLD IMAGES to be stored somewhere. Patching the enterprise applications needs considerable amount of patch download space and prorogation on regular intervals. This also need to be considered.

The other thing to consider is about the periodic tasks (jobs) that get executed to perform routine operations. Each job may produce few KB of output that need to be collect and stored in the central store for a period of time.

The next thing to consider is the number of users that use the application, number of concurrent users and the amount of data they retrieve from the data store for performing their operations.

Collecting all this information, adding the block header, segment header etc., overheads in the phisical structure of the target solution is a complex task. By the time this exercise is almost complete, the tool vendor would have released the next version of the data center management tool with some modifications!!

The solution:
It is nearly impossible to "size" any application to the last byte accuracy for storage, memory, Network and cpu utilization.

No managed entity should generate more than 25MB of data in the monitoring store. No managed entity should have more then 100kb of configuration data.
So, taking a 250GB storage for 1000 monitored entities and configuring the solution in such a way that it will not exceed this requirement is a wise man's solution.

Considering a 1000 monitored entities will have a maximum 100 administrators and at most 30 of them concurrently loggedin, An average db oltp application with 250GB database and 24 * 4 (considering 4 txn's an hour i.e., 15 min collection frequency) * 1000 (no of entities) would require a 2 processor DB server and a 2 processor Application server with 4GB RAM each.

Starting with this configuration, implementing the solution using an iterative model is the best approach to balance between the up-front sizing vs not impacting the service level of this critical service.

Conclusion:

In the current world with virtualization and cloud technology it is easier to build scalable Grid like applications and scaling the solutions horizontally is the trend that is going to stay and further pick-up more momentum. The data center management solutions are not exempt from this trend. For every next 1000 monitored entities we will have to add a node to the DB grid and another node to the AppServer Grid along with additional storage to the storage grid.

IS IT NOT??

Improved Application Performance Management

In April 2008 I have written this post : Application Performance Management

With
a. Oracle Enterprise Manager 10.2.0.5 Service Level Management,
b. REUI 6.0,
c. SOA mamagement pack and
d. Diagnostic Pack for middleware the solution is available.

A corporate can now implement a completely integrated APM solution based on the above four components of Oracle Enterprise Manager.

The new capabilities of REUI 6.0 include:
a. Full capture and replay of end user sessions.
b. Ability to link the slow performing JSP to the overall configuration in SOA management pack and then jump directly into the diagnostic tool (AD4J)
c. Customizable dashboards from the collected data.

What is REUI? "Real End User Experience Insight"! and What is 6.0? The new version released on 1st December 2009.

The ACTIVE MONITORING functionality is already provided by Enterprise Manager Service Level Management pack.

We hope these new tools will truly improve the Real End User Experience of the web applications.

Exadata V2 - World's First OLTP database machine

Last year the Exadata Database machine was announced for making the data warehouse performance improvement by moving the database query processing to the storage layer.

This year, the new Exadata V2 is proven to be the fastest database machine for OLTP applications.

The trick lies in the Flash solid state storage and the 40GBPS infiniband internal network fabric used between the DB servers and the Storage servers. The full rack contains 8 database servers and 14 storage servers. There are other configurations available.

All the flash storage (600 GB per storage server) is viewed as the extended memory of the servers and it has 72GB of DDR3 memory as the cache per database server....

Guess what, an open challenge to IBM can be found at http://www.oracle.com/features/exadatachallenge.html

So, why not try out for a $10 Million prize?