Upgrade pitblade from jdbc to das
From PTAGISWiki
Need to install DAS on pitblade and then retest with the all-sites query to see if it fails the same way.
Install DAS with these clues:
http://ingres.com/downloads/prod-cert-download.php Ingres 2006 Enterprise Edition GA Ingres 2006 Sun Solaris x86_64 (32-bit & 64-bit) RDBMS Executable 100MB-TAR II 9.0.4 (a64.sol/105) You just need to install just Ingres Networking to install Ingres client with DAS server. You can have your developer refer to the 'Installation Guide' -> 'Appendix A: Installing Ingres on UNIX'.
Download
The version of Ingres seems to have been updated since those instructions were written. The version I downloaded was this:
Ingres 2006 Release 2 Sun Solaris x86_64 (32-bit & 64-bit) RDBMS Executable 98MB-TAR II 9.1.0 (a64.sol/123) fe43ac9add74650fb72b914f671de12c
Following the directions at http://docs.ingres.com/install/InstallingIngresonUNIX
- Copied the downloaded tar to /usr/ingres/jdbc2das
- Untarred
Check current install with ingbuild
Checked the current installation on pitblade with /dsk1/AdvIngres/ing26/ingres/install/ingbuild It reports this:
Ingres Networking II 2.6/0207 (su4.us5/02) Ingres JDBC Driver and JDBC Server II 2.6/0207 (su4.us5/02)
Does that mean we already have the Ingres client with DAS server?
Trying to install with ingbuild
When I give ingbuild the path to the downloaded installation files, it says that Ingres Networking is ready to install. I don't have the option to say yes or no to installing Ingres Networking because it is fixed in the Ready state. Do I have to shut down ingres before this procedure?
Trying to install with the downloaded install.sh
Executing the install.sh that came in the tar, it says it must be run as root. When I run as root, it doesn't see the system environment variables and so thinks the ingres install should be in /opt/Ingres/IngresII.
How much of our Ingres environment needs to be set up before the install can work?
More Help From Ingres Tech Support
Divya determined that the tar I got was for the wrong architecture and so will not run on pitblade.
I downloaded this one next:
Ingres 2006 Release 2 Service Pack 1 Sun Solaris SPARC (32-bit and 64-bit) RDBMS Executable 119MB-TAR II 9.1.1 (su9.us5/103) 7c1234104b1cc142a98c5e8b7b6bf6ed
First I ran ingstop to bring the ingres install down.
After I untarred that file, I found another tarball inside it: ingres.tar I switched to $II_SYSTEM/ingres and extracted just the install directory from ingres.tar.
Then I could begin the installation with "install/ingbuild"
It didn't have the proper termcap setting, so I had to set vt100fx.
PackageInstall
I input the path to /dsk1/AdvIngres/ing26/ingres/install and selected PackageInstall.
That gave me the following options for installation:
- Full Installation
- Ingres Networked DBMS server
- Ingres Networked DBMS server w/STAR
- Ingres Networked client
- Ingres Stand-alone DBMS server
I chose Ingres Networked client since that seems closest to the directions to "install just Ingres Networking to install Ingres client with DAS server"
The installation seemed to start, but I also got the message:
pax: Type "go" when ready to proceed (or "quit" to abort):
It is only moving 91MB and it has taken about 2 minutes so far. After 10 minutes, the progress baton was still twirling and gave no sign of stopping. The installer is now stuck as far as I can tell. It is unresponsive to mouse clicks or key presses.
I issued a couple of ctrl-C's to get out of the installer.
I moved ingres/files/install.dat to install.dat.backup
Then I restarted ingbuild.
I again entered the path to the install directory and chose packageInstall.
I selected only the Ingres Networked Client and started the install. Again, I got the pax message:
Ready for volume 2m Distribution Medium Type "go" when ready to proceed (or "quit" to abort)
If I issue ctrl-C then I get a pax failed with exit code 130 This doesn't look like a tar archive.
Then a few returns and another ctrl-c gets me out.
CustomInstall
This time, I tried again but selected CustomInstall and now I have an option for "Ingres Networking". I set the install option to No for everything except for the Ingres Networking component. Then started the install.
Again I got the pax error:
pax: ready for volume 2all from Distribution Medium pax: Type go when ready to proceed or quit to abort.
After ctrl-c it says "this doesn't look like a tar archive"
different install path
This time I tried again but set the installation path to point to the ingres.tar file that was extracted from the downloaded tarball: /usr/ingres/jdbc2das/ingres2006-9.1.1-103c-com-sun-solaris-sparc-32-64bit/ingres.tar
That worked. I got one confirmation and I started the install. Then I got another confirmation asking to verify that ingres is shut down. I continued and it said it was done and asked if I want to setup the programs now.
I said yes. It asked if I really needed networking, I said yes. It asked if I want to create an installation password for this server, I gave it the development password. Then it said I was done and could issue the ingstart command.
ingstart
Now when I issue ingstart I get this error:
bash-2.05$ ingstart Ingres 2006 Release 2/ingstart Checking host "pitblade" for system resources required to run Ingres 2006 Release 2... 344064 byte shared memory segment required by LG/LK sub-systems. 0 bytes is the maximum shared memory segment size. 5 semaphores required. 0 is the total number of semaphores allocated by the system. 0 semaphores are currently available. 1 semaphore sets required. 0 is the total number of semaphore sets allocated by the system. 0 semaphore sets are currently available. Your system may not have sufficient resources to run Ingres 2006 Release 2 as configured.
This is a problem that Doug has seen before and come up with a workaround. I think it has to do with a command used to test for resources that doesn't work under Solaris and that he just commented it out of the the start script.
- I believe I first encountered the 'resource checking problem' ('$II_SYSTEM/ingres/utility/syscheck' won't run) when installing Ingres2006R2 with Solaris 10 on 'sockeye'. In terms of making the install, there was more to the workaround than replacing 'syscheck' with a soft-link to /bin/true. Please click here for details - in particular see comments time-stamped Tue Mar 6 09:24:19 PST 2007. Actually, 'syscheck' has been around forever and runs fine on Solaris - under 'the right circumstances'(!). My suspicion is that 'syscheck' is a 32-bit executable that won't run when Solaris is booted to the 64-bit kernel (never mind that the executable came in a distribution advertised as 32-64 bit compatible). Doug 08:44, 25 January 2008 (PST)
Ingres starts but doesn't work
Now Ingres will start successfully, but will not function. Doug noticed this and asked what was happening. He pointed me to the documentation on sockeye for installing Ingres.
Updating the Ingres Installs wiki page
There is a wiki page documenting current Ingres Installations. As I try to gather the information to update that page, it appears that the Ingres version that I installed is wildly different from our other installations. Ingbuild on pitblade now says the only component installed is Ingres Networking and it is version II 9.1.1 (su9.us5/103). Before this installation it was 32-bit 'AdvantageIngres 2.6' II 2.6/0207 (su4.us5/02b).
CBF
The next step from Ingres Tech support is to use CBF to turn off the JDBC server by setting the instance to 0. But when I run CBF, I don't see JDBC server as an option. There is just a name server and a DBMS server.
Advice from tech support
I told Divya that I didn't have the options in CBF that she asked me to modify. Her response was this:
Hi Ryan, It does not seem your installation went through okay. You seemed to have complete DBMS setup and then you upgraded just the networking component? From cbf it does not seem like you really are using DBMS (startup count 0). Do you need to use it? If not, can you scratch the installation and redo just the networking part? Right now I see a mix of Ingres 2.6 DBMS and Ingres 2006R2 Networking - Excerpt from config.dat showing this : ii.pitblade.config.dbms.ii260201su4us500: complete ii.pitblade.config.dbms.ii260207su4us502: complete ii.pitblade.config.jdbc.ii260201su4us500: complete ii.pitblade.config.jdbc.ii260207su4us502: complete ii.pitblade.config.net.ii260201su4us500: complete ii.pitblade.config.net.ii260207su4us502: complete ii.pitblade.config.net.ii911su9us5103: complete ii.pitblade.ingstart.*.dbms: 0 ii.pitblade.ingstart.*.gcc: 1 ii.pitblade.ingstart.*.jdbc: 1 ii.pitblade.ingstart.*.rmcmd: 1 It should look something like : ii.pitblade.config.gcd.ii911su9us5103: COMPLETE ii.pitblade.config.net.ii911su9us5103: COMPLETE ii.pitblade.ingstart.*.gcb: 0 ii.pitblade.ingstart.*.gcc: 1 ii.pitblade.ingstart.*.gcd: 1 ii.pitblade.ingstart.*.rmcmd: 0 cbf should have the following components only - +------------------------+------------------------+---------------+ |System Component |Configuration Name |Startup Count | |========================+========================+===============| |Name Server |(default) |1 | |Security |(default) |1 | |Net Server |(default) |1 | |Data Access Server |(default) |1 | |Bridge Server |(default) |0 | Let me know if you can do that. To install Ingres Networking only, you need to bring ingres down, make sure none of the ingres processes is running. Remove/ rename the directories under your current $II_SYSTEM/ingres. Follow the same process then as we discussed the other day to install Ingres networking.
She asks if I really need to use DBMS. I have no idea how to answer that.
She asks if I can scratch the current installation and just install the networking part. That sounds like she is asking me to upgrade from 2.6 to 2006R2. I thought I was just installing a new network device driver.
fresh install
- ingstop
- move $II_SYSTEM/ingres to a backup name (ingres_old)
- perform another CustomInstall
- cd $II_SYSTEM
- mkdir ingres
- cd ingres
- tar xvf /usr/ingres/jdbc2das/ingres2006-9.1.1-103c-com-sun-solaris-sparc-32-64bit/ingres.tar
- install/ingbuild
- set term to vt100fx
- set Distribution to /usr/ingres/jdbc2das/ingres2006-9.1.1-103c-com-sun-solaris-sparc-32-64bit/ingres.tar
- select CustomInstall
- set Ingres Networking to Yes
- set Ingres Data Access Server to Yes
- set all other options to No
- Configure Ingres networking? Yes
- installation code? II
- character set? accept default ISO88591
- time zone? NORTH-AMERICA
- region? NA-PACIFIC
- create an authorization entry? yes
- authorize all users on pitblade? no
- server host machine? sockeye
- server's listen address? UI
- installation password? <password>
- installation successful
- configure DAS? yes
- successfully setup
ingstart output
bash-2.05$ ingstart
Ingres 2006 Release 2/ingstart
Checking host "pitblade" for system resources required to run Ingres 2006 Release 2...
Your system has sufficient resources to run Ingres 2006 Release 2.
Starting your Ingres 2006 Release 2 installation...
Starting the Name Server...
WARNING!!! The recommended file descriptor hard limit for the NET Server to service 64 inbound,
and 64 outbound connections is 261. The current file descriptor hard limit is set to 256.
Starting Net Server (default)...
GCC Server = 50117
TCP_IP port = II (21064)
Starting Data Access Server (default)...
Ingres 2006 Release 2 installation successfully started.
cbf output
Now when I issue the command:
TERM_INGRES=vt100fx cbf
I get this output:
| System Component | Configuration Name | Startup Count |
| Name Server | default | 1 |
| Security | default | 1 |
| Net Server | default | 1 |
| Data Access Server | default | 1 |
testing with weblogic
- edit pitblade:/dsk2/local/bea81/domains/mydomain/startWeblogic.sh
- add iijdbc.jar to classpath
- restart weblogic
- edit connection url in weblogic admin console
- URL becomes: jdbc:ingres://localhost:II7/sockeye::ptagis3
- classname becomes: com.ingres.jdbc.IngresDriver
- restart weblogic
Got this error in weblogic STDOUT:
<Jan 28, 2008 4:01:14 PM PST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "PTAGIS3_Blueback": User provided a vnode as part of the database name (vnode::dbname), but connection information for that vnode is missing. Enter connection information for the vnode using NETUTIL.>
This is a red herring. The Blueback connection pool never was fully configured and always gave that error.
webapp config
SREE db.properties
- made edits to ptagis/WEB-INF/db.properties
- URL becomes: jdbc:ingres://localhost:II7/sockeye::ptagis3
- classname becomes: com.ingres.jdbc.IngresDriver
- made edits to sree/WEB-INF/db.properties
- URL becomes: jdbc:ingres://localhost:II7/sockeye::ptagis3
- classname becomes: com.ingres.jdbc.IngresDriver
- restarted weblogic
some things work
- Data > Reports > Adult Tagging Summary graph is empty
- Data > Reports > PTAGIS Codes works properly
- Data > Reports > New Complete Tag History works properly
- Data > Sites > Juvenile Interrogation > <anysite> has a broken graph
It appears that SREE hasn't fully gotten the news that it should use DAS.
SREE sree.properties
Ahah, the SREE scheduler has it's own classpath as seen here in sree.properties:
scheduler.classpath=/dsk2/ptagis-1.0/web/sree/WEB-INF/lib/sree_pro.jar\: /dsk2/ptagis-1.0/web/sree/WEB-INF/lib/etools.jar\: /dsk1/AdvIngres/ing26/ingres/lib/edbc.jar\: /dsk2/ptagis-1.0/web/sree/WEB-INF/classes\: /dsk2/ptagis-1.0/web/sree/WEB-INF/classes/ptagis\: /dsk2/ptagis-1.0/web/ptagis/WEB-INF/lib/commons-httpclient.jar\: /dsk2/ptagis-1.0/web/ptagis/WEB-INF/lib/commons-logging.jar
I added the following path:
:/dsk1/AdvIngres/ing26/ingres/lib/iijdbc.jar
Still, SREE doesn't seem to be fully functional.
- Data > Reports > Adult Tagging Summery is blank, but that gets generated and cached, so it isn't a good test. It will probably start to appear a day after SREE is otherwise working properly.
- Data > Sites > Juvenile Interrogation > MCJ still has a broken graph.
- Running a query builder report works
- perhaps those site graphs are also generated daily and will appear later.
Ingres environment on pitblade
Updated the /usr/ingres/set_ing_install.sh script to include the new jar file:
# Add JDBC driver to CLASSPATH ... if [ -s $II_SYSTEM/ingres/lib/iijdbc.jar ]; then CLASSPATH=$CLASSPATH:$II_SYSTEM/ingres/lib/iijdbc.jar export CLASSPATH fi
Missing some binaries
Doug discovered that the "sql" command is no longer recognized on pitblade. This is because ingres/bin/sql doesn't exist anymore. Doug says that the module I need from ingbuild is Query Reporting Tools.
- Installed Query Reporting Tools
- Installed Query Reporting Runtime
Both seemed to be related to qbf and rbf.
Now sql and qbf commands work.
jdbc info verification
bash-2.05$ java -cp .:./iijdbc.jar JdbcInfo jdbc:ingres://localhost:II7/sockeye::ptagis3 Ingres Corporation - JDBC Driver [3.2]: (JDBC 3.0) INGRES [9.1]: II 9.1.0 (su9.us5/123)
SREE config
datasource.xml
In addition to changes in sree.properties, database connection information is also kept in datasource.xml. At the top of the file are the stanzas that define the datasources. They look like this right now:
<datasource name="development" type="jdbc"> <ds_jdbc url="jdbc:edbc://localhost:II7/sockeye::p3_dev" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolation= "-1" ansiJoin="false" user="ptagdev" password="\pwd0e24050a223d25755c" tableTypes="TABLE,VIEW"> <description>Connects to p3_dev on sockeye as ptagdev</description> </ds_jdbc> </datasource> <datasource name="sockeye" type="jdbc"> <ds_jdbc url="jdbc:edbc://localhost:II7/sockeye::ptagis3" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolation ="-1" ansiJoin="false" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> </ds_jdbc> </datasource> <datasource name="production" type="jdbc"> <ds_jdbc url="jdbc:edbc://localhost:II7/blueback::ptagis3" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolatio n="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> </ds_jdbc> </datasource>
After the edits, those stanzas look like this:
<datasource name="development" type="jdbc"> <ds_jdbc url="jdbc:ingres://localhost:II7/sockeye::p3_dev" driver="com.ingres.jdbc.IngresDriver" requireLogin="true" transactionIso lation="-1" ansiJoin="false" user="ptagdev" password="\pwd0e24050a223d25755c" tableTypes="TABLE,VIEW"> <description>Connects to p3_dev on sockeye as ptagdev</description> </ds_jdbc> </datasource> <datasource name="sockeye" type="jdbc"> <ds_jdbc url="jdbc:ingres://localhost:II7/sockeye::ptagis3" driver="com.ingres.jdbc.IngresDriver" requireLogin="true" transactionIs olation="-1" ansiJoin="false" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> </ds_jdbc> </datasource> <datasource name="production" type="jdbc"> <ds_jdbc url="jdbc:ingres://localhost:II7/blueback::ptagis3" driver="com.ingres.jdbc.IngresDriver" requireLogin="true" transactionI solation="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> </ds_jdbc> </datasource>
individual .srt files
The jdbc url and driver class name are present in each .srt file that defines a query's data source. There are a lot of these. But not every .srt has this connection info in it. In /dsk2/ptagis-1.0/web/sree/WEB-INF/classes/ptagis there are 58 .srt files. Of those, there are 9 files that reference the EdbcDriver:
OSC_Site_Configuration.srt Site_Detection_Summary_by_Date_dd.srt Site_Summary_by_Date.srt Site_Summary_by_Date_dd.srt Site_Summary_for_smp.srt TAG_FILE_Dupe_Report.srt day_of_detection.srt huc-test.srt rkm.srt
Updated each of these report templates and restarted weblogic. OSC_Site_Configuration seems to work fine. But it might have never been broken. The site summary is still broken and the tagging, adult interrogation, juvenile interrogation are still broken.
stylereport.srl
This file is a zipped archive including beans, metareports, parameter sheets, scripts, styles. I found a parameter sheet that includes the jdbc url and class name. So this file will have to be modified as well.
templates directory
Found some references to the jdbc url in /dsk2/ptagis-1.0/web/sree/WEB-INF/classes/templates:
bash-2.05# grep edbc * Day of Detection.srt:<ds_jdbc url="jdbc:edbc://localhost:II7/blueback::ptagis3" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolation="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> OSC_Site_Configuration.srt:<ds_jdbc url="jdbc:edbc://localhost:II7/sockeye::ptagis3" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolation="-1" ansiJoin="false" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> SSBD-1212b-AB1.srt:<ds_jdbc url="jdbc:edbc://localhost:II7/blueback::ptagis3" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolation="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> day of detection.srt:<ds_jdbc url="jdbc:edbc://localhost:II7/blueback::ptagis3" driver="ca.edbc.jdbc.EdbcDriver" requireLogin="true" transactionIsolation="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW">
Updated these references and restarted weblogic. Graphs are still broken.
added production vnode in netutil
Under the assumption that we want to write a replet and test it on pitblade and deploy it to production without making any changes to the datasource, I set up a vnode on pitblade called production that is really just a copy of the sockeye vnode. That is, it points to sockeye, not blueback. I was prompted to do this by this error in sree.log:
[ERROR] Jan 30, 2008 3:59:45 PM: Connection Pool failed to create a new connection for: production [ERROR] Jan 30, 2008 3:59:45 PM: User provided a vnode as part of the database name (vnode::dbname), but connection information for that vnode is missing. Enter connection information for the vnode using NETUTIL.
That shows up when you try to view a site summary graph. But it still shows up ever after I created the production vnode and restarted weblogic.
following the code
The failed graph that I'd like to have working first is the site summary, like this one: http://pitblade.psmfc.org/sree/siteoverview.jsp?site_code=GOJ&site_type=obs
The code in siteoverview.jsp tells me that the replet of interest is public/Site Day of Detection Chart. That replet is defined in /ptagis/day_of_detection.srt
The datasource for that replet is defined in this code:
<datasource name="production" type="jdbc"> <ds_jdbc url="jdbc:ingres://localhost:II7/blueback::ptagis3" driver="com.ingres.jdbc.IngresDriver" requireLogin="true" transactionIsolation="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> </ds_jdbc> </datasource>
Checking the datasource.xml, I find that the definition for production looks like this:
<datasource name="production" type="jdbc"> <ds_jdbc url="jdbc:ingres://localhost:II7/blueback::ptagis3" driver="com.ingres.jdbc.IngresDriver" requireLogin="true" transactionIsolation="-1" ansiJoin="false" defaultDB="" user="websr" password="\pwd022e040b376020" tableTypes="TABLE,VIEW"> </ds_jdbc> </datasource>
So I think that the sree datasource called "production" is looking for a vnode called "blueback" on the localhost (pitblade). That fixed the problem. Now I have real data showing up in those graphs.
