Understanding The HTTP Plugin

I just found two great resources on the WebSphere plugin

• Demystifying the IBM® Web Server Plug-in 
• by Naveen Shetty AND Steve Reid
• Understanding the WebSphere Application Server Web server plug-in
• by Sharad Cocasse And Makarand Kulkarni

manageprofiles.bat and the zipProfile.ant

I recently backed up all of our WebSphere profiles using a home grown PowerShell script and in the process learned a few things about the manageprofiles tool.

Check the profile root for extras ! 

The first time I ran the process on the deployment manager profile it took around 10 to 20 min. When I moved to the profile that the application ran in I was surprised to see the process run for over an hour ! Turns out we had a bunch of .PHD files totaling a few Gigabytes. After I removed the .PHD files the process ran in 20 min.

zipProfile.ant is a trouble maker !

I got two errors that complained about the zipProfile.ant:13: saying  "The process cannot access the file because another process has locked a portion of the file." Turns out that this can be the cause of a few different things I experienced two.  The first was in one of  our testing environments. The appServerRoot>/properties/profileRegistry.xml had a profile listed isAReservationTicket="true". I set this to false (see the IBM notes below ) and the process ran as expected.

Zombies are never a good thing 

The second error was during the PRD release. After checking the profileRegistry.xml to make sure the isAReservationTicket value was not the issue, I checked my processes to see if I had a zombie to kill. Turns out my zombie stuck out like a sore thumb because we use service accounts to run our WebSphere services. 

Finale Thought

Lastly, I wanted to share how fast these error occur. If I had issues they always happened during the first few minutes of the process running.

Content and URL from the IBM KA.
Problem(Abstract)
manageprofiles.sh -backupProfile -profileName <your_profile> -backupFile <your_backup>
Symptom
The command fails with the following message:

Profile <your_profile> is currently in use: Retry the command later.  If there are no other processes are operating on the profile, then the profile might be corrupt.  Run the validateAndUpdateRegistry command and create the profile again.  INSTCONFFAILED: Cannot backup profile: For more information, consult /WebSphere/AppServer/logs/manageprofiles/your_profile_backupProfile.log.


Cause
It is possible a started process locked an XML file or if the process was killed, a zombie process was created that is holding a file lock.
Resolving the problem
Try the following 3 suggestions to help resolve the problem:
You can try rebooting the server. This will clear any zombie processes and any file locks they are holding.

If you cannot reboot the server, then you can try using the "lsof" command to list the open files on the server. Then look for any IBM® WebSphere® Application Server XML file for which there is an open file descriptor. You should be able to identify the process that is referencing that file descriptor and kill/close that process.

The original process may have create a lock file in the OS /tmp directory. Look for any files in the /tmp directory that were created around the time of the first manageprofiles session that you killed. Then remove that file.

If the preceeding 3 suggestions do not resolve the problem, try modifying the profileRegisry.xml file located in the following default path: 

<appServerRoot>/properties/profileRegistry.xml

Note: This will backup the file prior to making any manual changes. 

In this file there will be an entry corresponding to each profile. For example: 

<profile isAReservationTicket="true" isDefault="false" name="your_profile"
path="/WebSphere/AppServer/profiles/<your_profile>"
template="/WebSphere/AppServer/profileTemplates/default"/>

Change the value of isAReservationTicket="true" to isAReservationTicket="false". For example: 

<profile isAReservationTicket="false" isDefault="false" name="your_profile"
path="/WebSphere/AppServer/profiles/<your_profile>"
template="/WebSphere/AppServer/profileTemplates/default"/>


http://www-01.ibm.com/support/docview.wss?uid=swg21320418  

BMXAA6518E - Service RULESMANAGER is not loaded

I got the following error after I rebuilt and deployed the maximo ear file. 

[ERROR] BMXAA6518E - Service RULESMANAGER is not loaded. See the enclosed exception for detail.

psdi.util.MXSystemException: BMXAA3757E - The ServiceStorage: RULESMANAGER service could not be configured.

Turns out I was rushing a bit too much  and ran the buildmaximoear.cmd instead of the 

rm-buildmaximoear.cmd. The rm-buildmaximoear.cmd updates key xml files in the build, merging the Rules Manager product into the Maximo ear.

Server Names in MAXSESSION table

We found this great blog post  that shows how to set up the server names for cluster members. But that's not the real gem in my eyes, it how to use a variable in the JVM args ${WAS_SERVER_NAME} so that you don't have to hard code each JVM name in every app server config.

Apache The Definitive Guide - Book Review

I just finished Apache The Definitive Guide by Ben Laurie & Peter Laurie and would highly recommend this book to anybody who is administrating an Apache or IBM HTTP Server. Apache The Definitive Guide is a ground up approach to teaching you about the Apache web server platform. After spending the last year moving away from a developer role and into a infrastructure role I learned a lot about the IBM HTTP Server  but still had a lot of questions. The Definitive guide tied a lot of concepts together and removed the once dense fog around the web server component of our infrastructure.

Fixing an invalid Bind account

We have been going through a bunch of password changes and mistakenly change the password on a account in one environment that broke another environment. One of the things I learned (other then never reuse accounts ! ) was that when the bind account is invalid, even the wasadmin account will not be able to access the console. The environment we were working in was development and needing a fix I tried to edit the profiles\Dmgr01\config\cells\Cell01\wim\config\wimconfig.xml and updated the value for bindPassword and the bind account to a value from another instance .  What made this interesting was the fact that I was able to use a encrypted password from a WebSphere 6.1 environment in a WebSphere 7.0 environment. I was also able to update the bind password value with a clear text copy, allowing me access to the console. After I entered the console I went to
Secure administration, applications, and infrastructure > Federated repositories > repository
 and was then able to re encrypt the password by reentering it in the bind password box, clicking apply and then save.

WebSphere System Management & Config

My study notes from the following redbook

System Management Overview
    Terminology

• Stand-alone server environment refers to a single server environment that is not managed as part of a cell. This is your only option for base and express version of WebSphere but Network Deployment can be configured as a stand-alone server. 
• Distributed server environment refers to a environment that has multiple servers that are managed by a deployment manger in the cell. 
• Manage Process refers to deployment manager nodes (and node agents) and application servers. 

Stand-alone Single Server

 Distributed Server Environment 

Java Management Extensions (JMX)

   WebSphere system management functions are based on JMX a standard way to expose management objects. JMX