Apache build on bay

From PTAGISWiki

Jump to: navigation, search

In order for Request Tracker to run, it needs a load of perl modules and an apache server that can deal with Mason perl templating. They recommend either mod_perl or fastcgi.

This tweaked version of apache (built with fastcgi module), will reside in the /opt/rt3 directory along with Request Tracker and will service just RT's needs. This will prevent perl from bloating our httpd processes that serve the production web site. And hopefully it will segregate failures between our production system and RT.

Contents

build RT

  • installed dozens of perl module dependencies
  • configured RT
  • installed and configured mySQL
  • intialized database
  • started in standalone mode

Found that to perform further configuration (LDAP authentication) we'll need to do a full install. The web server in standalone mode is not configurable.

Acquired sources

Build apache first

bash-3.00# ./configure --prefix=/opt/rt3/apache
configure: error: APR-util version 1.2.0 or later is required
  • build apr-util: requires matching apr (apache portable runtime)
  • build apr 
./configure
make
/opt/rt3/build/apr-1.2.8/libtool: line 5565: ar: command not found
  • add /usr/ccs/bin to path
  • build apr 
./configure
make
  • build apr-util 
bash-3.00# ./configure --with-apr=/opt/rt3/build/apr-1.2.8
** success **
  • build apache 
./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 
--with-apr=/opt/rt3/build/apr-1.2.8

** success **
make

After finding out that SSL is not enabled by default, I have to come back to revisit this and add 

--enable-ssl --enable-info

The info is something that I tried to use in debugging and found it was not turned on.

Build mod_fastcgi

  • unpack in /opt/rt3/build/mod_fastcgi-2.4.2
  • copy to /opt/rt3/build/httpd-2.2.3/src/modules/fastcgi
  • cp Makefile.AP2 Makefile
  • rerun configure in httpd top level: 
./configure --prefix=/opt/rt3/build --with-apr-util=/opt/rt3/build/apr-util-1.2.8 
--with-apr=/opt/rt3/build/apr-1.2.8 --activate-module=src/modules/fastcgi/libfastcgi.a

configure: error: unrecognized option: --activate-module=src/modules/fastcgi/libfastcgi.a

OK, I'll have to try the manual install from http://www.fastcgi.com/mod_fastcgi/INSTALL That document assumes you have an <apachedir>/src/modules directory, but I don't. I wonder if that was changed in apache2.

Modified the Makefile in the fastcgi distribution to look like this: 

#
#  Makefile for Apache2
#

builddir     = .

#top_dir      = /usr/local/apache2
top_dir      = /opt/rt3/build/httpd

top_srcdir   = ${top_dir}
top_builddir = ${top_dir}

include ${top_builddir}/build/special.mk

APXS      = apxs
APACHECTL = apachectl

#DEFS=-Dmy_define=my_value
#INCLUDES=-Imy/include/dir
#LIBS=-Lmy/lib/dir -lmylib
INCLUDES=-I /opt/rt3/build/httpd/include -I /opt/rt3/build/apr-util/include

all: local-shared-build

install: install-modules

clean:
        -rm -f *.o *.lo *.slo *.la

But I get this: 

bash-3.00# make
/opt/rt3/build/apr-1.2.8/libtool --silent --mode=compile gcc -g -O2    -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE   -I /opt/rt3/build/httpd/include -I /opt/rt3/build/apr-util/include -I/opt/rt3/build/httpd/srclib/pcre -I. -I/opt/rt3/build/httpd/os/unix -I/opt/rt3/build/httpd/server/mpm/prefork -I/opt/rt3/build/httpd/modules/http -I/opt/rt3/build/httpd/modules/filters -I/opt/rt3/build/httpd/modules/proxy -I/opt/rt3/build/httpd/include -I/opt/rt3/build/httpd/modules/generators -I/opt/rt3/build/httpd/modules/mappers -I/opt/rt3/build/httpd/modules/database -I/opt/rt3/build/apr-1.2.8/include -I/opt/rt3/build/apr-util-1.2.8/include -I/usr/local/include -I/opt/rt3/build/httpd/modules/proxy/../generators -I/opt/rt3/build/httpd/modules/ssl -I/opt/rt3/build/httpd/modules/dav/main -prefer-pic -c mod_fastcgi.c && touch mod_fastcgi.slo
mod_fastcgi.c: In function `init_module':
mod_fastcgi.c:270: error: `ap_null_cleanup' undeclared (first use in this function)
mod_fastcgi.c:270: error: (Each undeclared identifier is reported only once
mod_fastcgi.c:270: error: for each function it appears in.)
mod_fastcgi.c: In function `process_headers':
mod_fastcgi.c:725: warning: return makes pointer from integer without a cast
mod_fastcgi.c:729: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c:739: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c:768: warning: initialization makes pointer from integer without a cast
mod_fastcgi.c:838: warning: return makes pointer from integer without a cast
mod_fastcgi.c:842: warning: return makes pointer from integer without a cast
mod_fastcgi.c: In function `set_uid_n_gid':
mod_fastcgi.c:1022: warning: passing arg 1 of `memcpy' makes pointer from integer without a cast
mod_fastcgi.c:1024: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c:1033: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c:1034: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c: In function `do_work':
mod_fastcgi.c:2321: error: `ap_null_cleanup' undeclared (first use in this function)
mod_fastcgi.c: In function `create_fcgi_request':
mod_fastcgi.c:2479: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c:2492: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c: In function `apache_is_scriptaliased':
mod_fastcgi.c:2534: warning: initialization makes pointer from integer without a cast
mod_fastcgi.c: In function `post_process_for_redirects':
mod_fastcgi.c:2559: warning: passing arg 1 of `ap_internal_redirect_handler' makes pointer from integer without a cast
mod_fastcgi.c: In function `check_user_authentication':
mod_fastcgi.c:2682: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c: In function `check_user_authorization':
mod_fastcgi.c:2749: warning: assignment makes pointer from integer without a cast
mod_fastcgi.c: In function `check_access':
mod_fastcgi.c:2809: warning: assignment makes pointer from integer without a cast
make: *** [mod_fastcgi.slo] Error 1

Googling turned up this from a forum: 

I could install the mod_fastcgi after patching the distribution using
the patch provided at following location.

http://www.fastcgi.com/archives/fastcgi-developers/2005-December/004060.html

This fixes the compilation problem "mod_fastcgi.c:271: error:
'ap_null_cleanup' undeclared" and allows to install using the
instructions in INSTALL.AP2,

Here is the patch: 

diff -ruN mod_fastcgi-2.4.2/fcgi.h mod_fastcgi-2.4.2-ap22/fcgi.h
--- mod_fastcgi-2.4.2/fcgi.h    2003-02-04 00:07:37.000000000 +0100
+++ mod_fastcgi-2.4.2-ap22/fcgi.h       2005-12-07 21:05:55.000000000 +0100
@@ -73,6 +73,36 @@
 #define ap_reset_timeout(a)
 #define ap_unblock_alarms()

+/* starting with apache 2.2 the backward-compatibility defines for
+ * 1.3 APIs are not available anymore. Define them ourselves here.
+ */
+#ifndef ap_copy_table
+
+#define ap_copy_table apr_table_copy
+#define ap_cpystrn apr_cpystrn
+#define ap_destroy_pool apr_pool_destroy
+#define ap_isspace apr_isspace
+#define ap_make_array apr_array_make
+#define ap_make_table apr_table_make
+#define ap_null_cleanup apr_pool_cleanup_null
+#define ap_palloc apr_palloc
+#define ap_pcalloc apr_pcalloc
+#define ap_psprintf apr_psprintf
+#define ap_pstrcat apr_pstrcat
+#define ap_pstrdup apr_pstrdup
+#define ap_pstrndup apr_pstrndup
+#define ap_push_array apr_array_push
+#define ap_register_cleanup apr_pool_cleanup_register
+#define ap_snprintf apr_snprintf
+#define ap_table_add apr_table_add
+#define ap_table_do apr_table_do
+#define ap_table_get apr_table_get
+#define ap_table_set apr_table_set
+#define ap_table_setn apr_table_setn
+#define ap_table_unset apr_table_unset
+
+#endif /* defined(ap_copy_table) */
+
 #if (defined(HAVE_WRITEV) && !HAVE_WRITEV && !defined(NO_WRITEV)) || defined WIN32
 #define NO_WRITEV
 #endif
diff -ruN mod_fastcgi-2.4.2/Makefile.AP2 mod_fastcgi-2.4.2-ap22/Makefile.AP2
--- mod_fastcgi-2.4.2/Makefile.AP2      2002-07-29 03:36:34.000000000 +0200
+++ mod_fastcgi-2.4.2-ap22/Makefile.AP2 2005-12-07 20:27:50.000000000 +0100
@@ -20,8 +20,6 @@

 all: local-shared-build

-install: install-modules
-
 clean:
        -rm -f *.o *.lo *.slo *.la

After applying the patch, the make worked.

Copied the resulting module (mod_fastcgi.so) into httpd/modules and add this stanza to httpd.conf: 

      LoadModule fastcgi_module modules/mod_fastcgi.so

Build apache again

 bash-3.00# ./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 
--with-apr=/opt/rt3/build/apr-1.2.8

make
make install

start apache

/opt/rt3/apache/bin/apachectl start
httpd: Could not open configuration file /opt/rt3/build/conf/httpd.conf: No such file or directory

Why is it trying to open files in the build directory? Did I forget to do a make clean at some point? The config file is right where it should be: /opt/rt3/apache/conf/httpd.conf

Did another cycle of make clean; ./configure*; make; make install and it worked.

configure rt

I created a configuration stub for apache called rt.conf with these contents as per the RT book: 

<VirtualHost *>
        ServerName www.ptagis.org
        ServerAdmin webmaster@psmfc.org
        FastCgiServer /opt/rt3/bin/mason_handler.fcgi
        AddHandler fastcgi-script fcgi
        Alias /NoAuth/images/ /opt/rt3/html/NoAuth/images/
        ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/
</VirtualHost>

But apache doesn't like this syntax: 

Syntax error on line 4 of /opt/rt3/apache/conf/conf.d/rt.conf:
FastCgiServer cannot occur within <VirtualHost> section

Moved the FastCgiServer up and out of the virtual host section.

Had to change the permissions to allow from all in the standard httpd.conf

Now I get this error in the apache log: 

[Tue Dec 19 11:21:48 2006] [warn] FastCGI: server "/opt/rt3/bin/mason_handler.fcgi" (pid 11382) terminated by calling exit with status '13'
[Tue Dec 19 11:21:51 2006] [warn] FastCGI: server "/opt/rt3/bin/mason_handler.fcgi" restarted (pid 11384)
Can't locate /opt/rt3/etc/RT_SiteConfig.pm in @INC (@INC contains: /opt/rt3/local/lib /opt/rt3/lib /usr/local/lib/perl5/5.8.7/sun4-solaris /usr/local/lib/perl5/5.8.7 /usr/local/lib/perl5/site_perl/5.8.7/sun4-solaris /usr/local/lib/perl5/site_perl/5.8.7 /usr/local/lib/perl5/site_perl .) at /opt/rt3/lib/RT.pm line 131.
BEGIN failed--compilation aborted at /opt/rt3/bin/webmux.pl line 76.
Compilation failed in require at /opt/rt3/bin/mason_handler.fcgi line 52.
[Tue Dec 19 11:21:53 2006] [warn] FastCGI: server "/opt/rt3/bin/mason_handler.fcgi" (pid 11384) terminated by calling exit with status '13'
[Tue Dec 19 11:21:53 2006] [warn] FastCGI: server "/opt/rt3/bin/mason_handler.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to 600 seconds
[Tue Dec 19 11:22:15 2006] [error] [client 10.1.2.59] FastCGI: comm with server "/opt/rt3/bin/mason_handler.fcgi" aborted: idle timeout (30 sec)
[Tue Dec 19 11:22:15 2006] [error] [client 10.1.2.59] FastCGI: incomplete headers (0 bytes) received from server "/opt/rt3/bin/mason_handler.fcgi"

Actually, the previous RT config was done for the standalone server. The RT install was never completed for a real installation. So the next steps in getting RT working will be back in the Issue tracker article.

more web debugging

After going as far as possible on RT install and config, I'm trying to get something to start just in case the install actually worked. The install hung up when it said that DBD::mysql was missing even though it is installed.

The same errors above are present in the apache log when I try to connect to the RT web gui. Running mason_handler.fcgi from the command line, I get this on stdout: 

bash-3.00# ./mason_handler.fcgi
install_driver(mysql) failed: Can't load '/usr/local/lib/perl5/site_perl/5.8.7/sun4-solaris/auto/DBD/mysql/mysql.so' for module DBD::mysql: ld.so.1: perl: fatal: libmysqlclient.so.15: open failed: No such file or directory at /usr/local/lib/perl5/5.8.7/sun4-solaris/DynaLoader.pm line 230.
 at (eval 199) line 3
Compilation failed in require at (eval 199) line 3.
Perhaps a required shared library or dll isn't installed where expected
 at /usr/local/lib/perl5/site_perl/5.8.7/DBIx/SearchBuilder/Handle.pm line 106

More evidence that DBD::mysql is not installed properly.

LD_LIBRARY_PATH

After setting LD_LIBRARY_PATH to include the mysql libs I get a clean run of mason_handler.fcgi: 

bash-3.00# ./mason_handler.fcgi
Pragma: no-cache
Set-cookie: RT_SID_ptagis.org.=6871201f52bd859da3fc40165f305e2c; path=/
Cache-control: no-cache
Content-Type: text/html; charset=utf-8



<!DOCTYPE html
     PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    <title>Login</title>
    <link rel="shortcut icon" href="//NoAuth/images//favicon.png" type="image/png" />
    <link rel="stylesheet" href="/NoAuth/css/3.5-default/main.css" type="text/css" media="all" />
    <link rel="stylesheet" href="/NoAuth/css/print.css" type="text/css" media="print" />
    <script type="text/javascript" src="/NoAuth/js/util.js"></script>

    <script type="text/javascript" src="/NoAuth/js/ahah.js"></script>
    <script type="text/javascript" src="/NoAuth/js/titlebox-state.js"></script>
    <script type="text/javascript"><!--
        onLoadHook("loadTitleBoxStates()");
        onLoadHook("focusElementById('user')");
    --></script>

  </head>
  <body id="comp-index">

  <div id="logo">
    <a href="http://bestpractical.com"><img src="//NoAuth/images//bplogo.gif" alt="Best Practical Solutions, LLC corporate logo" width="177" height="33" /></a>
    <div class="rtname">RT for ptagis.org</div>
  </div>


<div id="quickbar">
  <div id="quick-personal">
    <span class="hide"><a href="#skipnav">Skip Menu</a> | </span>
    Not logged in.

  </div>




</div>

<div id="body" class="login-body">




<div id="login-box">
<div class="">
  <div class="titlebox " id="">
  <div class="titlebox-title">
    <span class="left">Login</span>
    <span class="right">3.6.2</span>
  </div>
  <div class="titlebox-content " id="TitleBox--_index.html------Login---0">




<form id="login" name="login" method="post" action="index.html">

<div class="input-row">
    <span class="label">Username:</span>
    <span class="input"><input name="user" value="" id="user" /></span>
</div>

<div class="input-row">
    <span class="label">Password:</span>
    <span class="input"><input type="password" name="pass" /></span>
</div>

<div class="button-row">
    <span class="input"><input type="submit" class="button" value="Login" /></span>
</div>



</form>
    <hr class="clear" />
  </div>
</div>




</div>


</div><!-- #login-box -->

</div>

<div id="footer">
  <p id="time">
    <span>Time to display: 0.208915</span>
  </p>

  <p id="bpscredits">
    <span>
»|« RT 3.6.2 Copyright 1996-2006 <a href="http://www.bestpractical.com?rt=3.6.2">Best Practical Solutions, LLC</a>.
</span>
</p>
  <p id="legal">
Distributed under version 2 <a href="http://www.gnu.org/copyleft/gpl.html"> of the GNU GPL.</a><br />
To inquire about support, training, custom development or licensing, please contact <a href="mailto:sales@bestpractical.com">sales@bestpractical.com</a>.<br />
  </p>

</div>

  </body>
</html>

apache config

Changed apache config to look like this: 

SetEnv LD_LIBRARY_PATH /usr/local/mysql/lib/mysql/
FastCgiServer /opt/rt3/bin/mason_handler.fcgi
<VirtualHost *>
        ServerName www.ptagis.org
        ServerAdmin webmaster@psmfc.org
        AddHandler fastcgi-script fcgi
        Alias /NoAuth/images/ /opt/rt3/html/NoAuth/images/
        ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/
</VirtualHost>

permissions error in RT_SiteConfig

Got this error in the apache log when trying to connect: 

Couldn't load RT config file /opt/rt3/etc/RT_SiteConfig.pm as user daemon / group daemon.
The file is owned by user root and group www.
This usually means that the user/group your webserver is running as cannot read
the file.
Be careful not to make the permissions on this file too liberal, because it contains database
passwords.  You may need to put the webserver user in the appropriate group (www) or change
permissions be able to run succesfully

Changed group ownership of everything in /opt/rt3/etc to daemon

still can't load libmysqlclient.so

I still get the error about libmysqlclient not being found. I've tweaked the apache config to look like this: 

SetEnv LD_LIBRARY_PATH /usr/local/mysql/lib/mysql/
PassEnv LD_LIBRARY_PATH
FastCgiIpcDir /tmp
FastCgiServer /opt/rt3/bin/mason_handler.fcgi
<VirtualHost *>
        ServerName www.ptagis.org
        DocumentRoot /opt/rt3/share/html
        ServerAdmin webmaster@psmfc.org
        AddHandler fastcgi-script fcgi
        Alias /NoAuth/images/ /opt/rt3/html/NoAuth/images/
        ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/
        <Location />
                AddDefaultCharset UTF-8
                SetHandler fastcgi-script
        </Location>
</VirtualHost>

But it still fails. The juiciest errors appear right when I try to start apache: 

install_driver(mysql) failed: Can't load '/usr/local/lib/perl5/site_perl/5.8.7/sun4-solaris/auto/DBD/mysql/mysql.so' for 
module DBD::mysql: ld.so.1: perl: fatal: libmysqlclient.so.15: open failed: 
No such file or directory at /usr/local/lib/perl5/5.8.7/sun4-solaris/DynaLoader.pm line 230.
 at (eval 199) line 3

Also modified envvars to look like this: 

# envvars-std - default environment variables for apachectl
#
# This file is generated from envvars-std.in
#
LD_LIBRARY_PATH="/opt/rt3/apache/lib:/usr/local/mysql/lib/mysql:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH

But it still can't find libmysqlclient.so

Tried turning on server-info and server-status to get more debug info, but apparently, they depend on fastcgi, since they only return "500 server error" and I get fastcgi errors in the log when I try to hit those pages.

exploring difference between command line and apache environment

Since I can get mason_handler.fcgi to run from the command line, but it fails when apache tries it, I tried whittling down my command line environment to see if I can get it to approach the environment that apache has.

I set the paths to this: 

CLASSPATH=/usr/ingres/ing26/ingres/lib/edbc.jar:
LD_LIBRARY_PATH=/usr/local/mysql/lib/mysql
PATH=/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/op_ing/ingII/bin:/usr/op_ing/ingII/utility:
/usr/op_ing/ingII/ingres/bin

and did an su to user daemon since that is the user apache runs as.

Then when I run mason_handler.fcgi, it still works. Huh. I can't think what else might be different.

I wish I could verify that the environment variables are being properly loaded by apache... if I need a plain cgi program that reports the current environment.

loaded modules

apachectl -M reports the following modules as being loaded: 


bash-3.00$ ./apachectl -M
Loaded Modules:
 core_module (static)
 authn_file_module (static)
 authn_default_module (static)
 authz_host_module (static)
 authz_groupfile_module (static)
 authz_user_module (static)
 authz_default_module (static)
 auth_basic_module (static)
 include_module (static)
 filter_module (static)
 log_config_module (static)
 env_module (static)
 setenvif_module (static)
 mpm_prefork_module (static)
 http_module (static)
 mime_module (static)
 status_module (static)
 autoindex_module (static)
 asis_module (static)
 cgi_module (static)
 negotiation_module (static)
 dir_module (static)
 actions_module (static)
 userdir_module (static)
 alias_module (static)
 so_module (static)
 fastcgi_module (shared)
Syntax OK

printenv results

Turned off the rt.conf part of apache and ran a sample cgi script to display the environment: 

DOCUMENT_ROOT="/opt/rt3/apache/htdocs"
GATEWAY_INTERFACE="CGI/1.1"
HTTP_ACCEPT="text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"
HTTP_ACCEPT_CHARSET="ISO-8859-1,utf-8;q=0.7,*;q=0.7"
HTTP_ACCEPT_ENCODING="gzip,deflate"
HTTP_ACCEPT_LANGUAGE="en-us,en;q=0.7,ko;q=0.3"
HTTP_CONNECTION="keep-alive"
HTTP_HOST="bay.psmfc.org:7777"
HTTP_KEEP_ALIVE="300"
HTTP_USER_AGENT="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.1) Gecko/20061208 Firefox/2.0.0.1"
PATH="/bin:/usr/bin:/usr/sbin"
QUERY_STRING=""
REMOTE_ADDR="10.1.2.59"
REMOTE_PORT="52824"
REQUEST_METHOD="GET"
REQUEST_URI="/cgi-bin/printenv"
SCRIPT_FILENAME="/opt/rt3/apache/cgi-bin/printenv"
SCRIPT_NAME="/cgi-bin/printenv"
SERVER_ADDR="199.170.103.41"
SERVER_ADMIN="admin@psmfc.org"
SERVER_NAME="bay.psmfc.org"
SERVER_PORT="7777"
SERVER_PROTOCOL="HTTP/1.1"
SERVER_SIGNATURE=""
SERVER_SOFTWARE="Apache/2.2.3 (Unix)"
TZ="US/Pacific"

It doesn't have LD_LIBRARY_PATH. The program looks like this: 

#!/usr/local/bin/perl
##
##  printenv -- demo CGI program which just prints its environment
##

print "Content-type: text/plain; charset=iso-8859-1\n\n";
foreach $var (sort(keys(%ENV))) {
    $val = $ENV{$var};
    $val =~ s|\n|\\n|g;
    $val =~ s|"|\\"|g;
    print "${var}=\"${val}\"\n";
}

Added this to envvars: 

TEST="testing"
export TEST

and it did not get picked up by apache. printenv still shows the same environment variables.

Here is the code in apachectl that mentions envvars: 

# pick up any necessary environment variables
if test -f /opt/rt3/apache/bin/envvars; then
  . /opt/rt3/apache/bin/envvars
fi

If I execute that code from the command line as user daemon, it works.

I was able to set the TZ variable from within envvars. TZ was an existing variable and I changed it from "US/Pacific" to "US/Pacific/bay" by setting it in envvars. Environment variables are probably getting wiped for security...

The "PassEnv TEST" directive had to be set in conjunction with the envvars file to get the TEST variable to show up: 

SERVER_SOFTWARE="Apache/2.2.3 (Unix)"
TEST="testing"
TZ="US/Pacific/bay"

So setting PassEnv LD_LIBRARY_PATH in the main httpd.conf actually worked. The apache no longer gives library errors when trying to run mason_handler.fcgi. Now it gives this error instead: 

[Tue Dec 26 23:40:52 2006] [error] [client 10.1.2.59] (2)No such file or directory: FastCGI: failed to connect to server "/opt/rt3/bin/mason_handler.fcgi": connect() failed
[Tue Dec 26 23:40:52 2006] [error] [client 10.1.2.59] FastCGI: incomplete headers (0 bytes) received from server "/opt/rt3/bin/mason_handler.fcgi"

There are no errors on apache startup. Just these errors each time I try to go to an RT URL.

The right way to pass environment variables

It seems there are several ways to set environment variables for apache:

  • PassEnv
this tells apache to look at the parent environment when apache is started and pick the named variables to hand down to the child processes
  • SetEnv
this tells apache to set variables that will be available to child processes
  • envvars (startup script)
this is a file that is included when apachectl is used to start apache and can be used to set environment variables

I was getting weird interaction between these different methods. When I used only the SetEnv directive, I got predictable results and was able to limit my config to one place.

However, getting LD_LIBRARY_PATH set doesn't get the mason_handler to run within apache...

apache vhost config tweaks

After trying to fiddle with the virtualhost directives for RT, I'm now stuck with the missing module errors again. And I'm not sure how to change it back.

Perhaps I never had the FastCgiServer directive in place. The RT book says that has to be in the virtual host, but apache tells me that is a syntax error and it needs to be in the server config context. The Fastcgi page tells me that is the directive to set up a static fastcgi server, but my module is dynamic. Maybe that is why it is griping. I may have to set up the fastcgi server to run as an independent process with its own port.

trying to rebuild mod_fastcgi with apxs

The bigadmin howto (http://www.sun.com/bigadmin/features/articles/req_track_1.html) for RT uses apxs to build the fastcgi module. I followed the build instructions from the fastcgi distribution that just used make. I'll try the apxs method here. 

bash-3.00# /opt/rt3/apache/bin/apxs -o mod_fastcgi.so -c *.c /opt/rt3/apache/bin/apxs -i -a -n fastcgi mod_fastcgi.so
/usr/local/apr/build-1/libtool --silent --mode=compile gcc -prefer-pic   -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -g -O2 -I/opt/rt3/apache/include  -I/usr/local/apr/include/apr-1   -I/usr/local/apr/include/apr-1 -I/usr/local/include  -c -o fcgi_buf.lo fcgi_buf.c && touch fcgi_buf.slo
In file included from /opt/rt3/apache/include/httpd.h:43,
                 from fcgi.h:22,
                 from fcgi_buf.c:5:
/opt/rt3/apache/include/ap_config.h:26:23: apr_hooks.h: No such file or directory

Many more compile errors are elided.

It appears to be looking in the apache/include for apr/include (apache runtime) files. Maybe I can just move them over: 

bash-3.00# pwd
/opt/rt3/build/apr/include
bash-3.00# cp apr* /opt/rt3/apache/include/

Now I get this: 

bash-3.00# /opt/rt3/apache/bin/apxs -o mod_fastcgi.so -c *.c /opt/rt3/apache/bin/apxs -i -a -n fastcgi mod_fastcgi.so
/usr/local/apr/build-1/libtool --silent --mode=compile gcc -prefer-pic   -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -g -O2 -I/opt/rt3/apache/include  -I/usr/local/apr/include/apr-1   -I/usr/local/apr/include/apr-1 -I/usr/local/include  -c -o fcgi_buf.lo fcgi_buf.c && touch fcgi_buf.slo
In file included from /opt/rt3/apache/include/httpd.h:43,
                 from fcgi.h:22,
                 from fcgi_buf.c:5:
/opt/rt3/apache/include/ap_config.h:26:23: apr_hooks.h: No such file or directory

And many more compile errors.

Guess I need to copy in the include files from apr-utils as well. Makes sense. 

bash-3.00# cp apr* /opt/rt3/apache/include/
bash-3.00# pwd
/opt/rt3/build/apr-util/include

Now I get: 

bash-3.00# /opt/rt3/apache/bin/apxs -o mod_fastcgi.so -c *.c /opt/rt3/apache/bin/apxs -i -a -n fastcgi mod_fastcgi.so
/usr/local/apr/build-1/libtool --silent --mode=compile gcc -prefer-pic   -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -g -O2 -I/opt/rt3/apache/include  -I/usr/local/apr/include/apr-1   -I/usr/local/apr/include/apr-1 -I/usr/local/include  -c -o fcgi_buf.lo fcgi_buf.c && touch fcgi_buf.slo
In file included from /opt/rt3/apache/include/ap_config.h:26,
                 from /opt/rt3/apache/include/httpd.h:43,
                 from fcgi.h:22,
                 from fcgi_buf.c:5:
/opt/rt3/apache/include/apr_hooks.h:20:17: apu.h: No such file or directory

There was a bug in my copy command. I changed it to copy *h instead.

Now it appears to be compiling correctly...

It completed without errors, but I don't see where it put the mod_fastcgi.so... 

find . -name mod_fastcgi.so

tells me that it is here: 

256626  216 -rwxr-xr-x   1 root     1172       205052 Dec 27 12:10 ./build/mod_fastcgi-2.4.2/.libs/mod_fastcgi.so

The bigadmin howto doesn't say anything about moving the shared library after installing it... But I'll copy it into apache/modules since that is what I tell apache to do in this directive: 

LoadModule fastcgi_module modules/mod_fastcgi.so

The apxs version of mod_fastcgi does not fix the problem. It still fails to find the libmysqlclient.so library.

running mason_handler.fcgi from envvars

Made this addition to envvars: 

/opt/rt3/bin/mason_handler.fcgi > /tmp/apachetruss.txt

and found that it worked just as well as running it from the command line, as long as the LD_LIBRARY_PATH was set properly. So perhaps it is something else that is giving this error. I need to find just where the error is coming from during the startup of apache.

solution: FastCgiServer -initial-env

FastCgiServer /opt/rt3/bin/mason_handler.fcgi -initial-env LD_LIBRARY_PATH

When I added that option to my FastCgiServer directive, I was able to start apache with no errors and when I went to the front page URL of RT, it worked. Amazing. The clue was in mason_handler.fcgi's code: 

    # the whole point of fastcgi requires the env to get reset here..
    # So we must squash it again

So when I was reading the options for the FastCgiServer directive, that one stood out.

openssl

SSL is a requirement. I thought it was optional. Now I must get it installed.

Did a vanilla download of latest openssl source, config, make, make test.

It ran without errors. The make test ran for several minutes.

Ran make install. It installed in /usr/local/ssl.

install mod_ssl for apache

The bigadmin howtow for RT uses apache 1.3 and mod_ssl. I'm using apache 2.2, but mod_ssl seems to be still the preferred solution according to this documentation: http://httpd.apache.org/docs/2.2/ssl/

Apparently mod_ssl is now part of the base of apache2 so there is not need to download anything extra. But I should have built apache with ssl support via "configure --enable-ssl". Bummer.

rebuild apache with ssl

Rebuilt apache with this configure: 

bash-3.00# ./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 --with-apr=/opt/rt3/build/apr-1.2.8 --enable-ssl --enable-info

But got this error: 

configure: error: ...No recognized SSL/TLS toolkit detected

Added this switch: 

--with-ssldir=/usr/local/ssl

But that didn't work. Tried the build directory for openssl: 

 --with-ssldir=/opt/rt3/build/openssl

Same error.

Here is the proper switch: 

--with-ssl=/usr/local/ssl

make
make install

make self-signed cert

cd /opt/rt3/apache/conf
/usr/local/ssl/bin/openssl genrsa
cut and paste into privkey.pem
/usr/local/ssl/bin/openssl req -new -x509 -key privkey.pem -out server.crt -days 3650
fill out form
cp privkey.pem server.key

start apache and it works

Now this url works: https://bay.psmfc.org:7777/

forgot ldap: rebuild apache again

How did I not see this in ./configure --help? 

  --enable-authnz-ldap    LDAP based authentication

Anyway, let's try this again: 

./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 --with-apr=/opt/rt3/build/apr-1.2.8 --enable-ssl --enable-info --enable-authnz-ldap --with-ssl=/usr/local/ssl

No dice. Searching the web I see someone used these three configure switches: 

--enable-authnz-ldap --with-ldap --enable-ldap

But that still doesn't work. I get this helpful message when trying to make. 

mod_authnz_ldap.c:40:2: #error mod_authnz_ldap requires APR-util to have LDAP support built in. To fix add --with-ldap to ./configure.

Rebuilt apr-util --with-ldap. make. make install.

Rebuilt apache: 

./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 --with-apr=/opt/rt3/build/apr-1.2.8 --enable-ssl --enable-info --enable-authnz-ldap --with-ssl=/usr/local/ssl --with-ldap --enable-ldap
make

Yields this error: 

Undefined                       first referenced
 symbol                             in file
apr_ldap_info                       server/.libs/libmain.a(exports.o)
apr_ldap_init                       server/.libs/libmain.a(exports.o)
apr_ldap_url_parse_ext              server/.libs/libmain.a(exports.o)
apr_ldap_ssl_deinit                 server/.libs/libmain.a(exports.o)
apr_ldap_is_ldapi_url               server/.libs/libmain.a(exports.o)
apr_ldap_is_ldaps_url               server/.libs/libmain.a(exports.o)
apr_ldap_set_option                 server/.libs/libmain.a(exports.o)
apr_ldap_get_option                 server/.libs/libmain.a(exports.o)
apr_ldap_is_ldap_url                server/.libs/libmain.a(exports.o)
apr_ldap_ssl_init                   server/.libs/libmain.a(exports.o)
apr_ldap_url_parse                  server/.libs/libmain.a(exports.o)
ld: fatal: Symbol referencing errors. No output written to .libs/httpd
collect2: ld returned 1 exit status

install openldap and rebuild apache

Perhaps the error is telling me that it needs openldap libraries. Googling for the key words in the configure command showed me that other people use these switches: 

--with-ldap-sdk=openldap \
--with-sdk-libs=/usr/local/lib \
--with-sdk-headers=/usr/local/include

Todd installed openldap and I tried again. 

./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 --with-apr=/opt/rt3/build/apr-1.2.8 --enable-ssl --enable-info --enable-authnz-ldap --with-ssl=/usr/local/ssl --with-ldap --enable-ldap --with-ldap-sdk=openldap

Gave the same error when doing the make. 

./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 --with-apr=/opt/rt3/build/apr-1.2.8 --enable-ssl --enable-info --enable-authnz-ldap --with-ssl=/usr/local/ssl --with-ldap --enable-ldap --with-ldap-sdk=openldap --with-sdk-libs=/usr/local/lib --with-sdk-headers=/usr/local/include

Darn, same error. 

Undefined                       first referenced
 symbol                             in file
apr_ldap_info                       server/.libs/libmain.a(exports.o)
apr_ldap_init                       server/.libs/libmain.a(exports.o)
apr_ldap_url_parse_ext              server/.libs/libmain.a(exports.o)
apr_ldap_ssl_deinit                 server/.libs/libmain.a(exports.o)
apr_ldap_is_ldapi_url               server/.libs/libmain.a(exports.o)
apr_ldap_is_ldaps_url               server/.libs/libmain.a(exports.o)
apr_ldap_set_option                 server/.libs/libmain.a(exports.o)
apr_ldap_get_option                 server/.libs/libmain.a(exports.o)
apr_ldap_is_ldap_url                server/.libs/libmain.a(exports.o)
apr_ldap_ssl_init                   server/.libs/libmain.a(exports.o)
apr_ldap_url_parse                  server/.libs/libmain.a(exports.o)
ld: fatal: Symbol referencing errors. No output written to .libs/httpd
collect2: ld returned 1 exit status
make[1]: *** [httpd] Error 1
make[1]: Leaving directory `/opt/rt3/build/httpd-2.2.3'
make: *** [all-recursive] Error 1

Tried a make clean, make on apr and apr-util to make sure that my configure options are being acted upon. Then did a make clean, make on httpd. did it work? It worked! 

bash-3.00# ./apachectl -M
Loaded Modules:
 core_module (static)
 authn_file_module (static)
 authn_default_module (static)
 authz_host_module (static)
 authz_groupfile_module (static)
 authz_user_module (static)
 authnz_ldap_module (static)
 authz_default_module (static)
 auth_basic_module (static)
 include_module (static)
 filter_module (static)
 ldap_module (static)
 log_config_module (static)
 env_module (static)
 setenvif_module (static)
 ssl_module (static)
 mpm_prefork_module (static)
 http_module (static)
 mime_module (static)
 status_module (static)
 autoindex_module (static)
 asis_module (static)
 info_module (static)
 cgi_module (static)
 negotiation_module (static)
 dir_module (static)
 actions_module (static)
 userdir_module (static)
 alias_module (static)
 so_module (static)
 fastcgi_module (shared)
Syntax OK

Ugh, now I get this when I try to start apache: 

ld.so.1: httpd: fatal: relocation error: file /opt/rt3/apache/bin/httpd: 
symbol apr_ldap_info: referenced symbol not found

rebuild apr-util with ldap

The ./configure --help for apr doesn't have any mention of LDAP, but the apr-util does. Here is what it says: 

  --with-ldap-include=path  path to ldap include files with trailing slash
  --with-ldap-lib=path    path to ldap lib file
  --with-ldap=library     ldap library to use

Rebuilding apr-util with these switches: 

./configure --with-apr=/opt/rt3/build/apr-1.2.8 --with-ldap-include=/usr/local/include --with-ldap-lib=/usr/local/lib --with-ldap=openldap

checking for ldap_init in -lopenldap... no
checking for ldap_init in -lopenldap... no
checking for ldap_init in -lopenldap... no
checking for ldap_init in -lopenldap... no
configure: error: could not find an LDAP library

I wonder if it is confused by this: 

bash-3.00# grep ldap_init /usr/local/include/ldap*
/usr/local/include/ldap.h:ldap_init LDAP_P(( /* deprecated */
/usr/local/include/ldap.h:ldap_initialize LDAP_P((

It is there, just deprecated. Odd... it failed with the switch --with-ldap=openldap, but it works with --with-ldap


Also, it didn't work to do a make clean on the apr, apr-util, and httpd directories and then just do a ./configure, make in httpd. I thought make might be smart enough to handle the dependencies better than I could.

So I ended up with this: 

./configure --with-ldap --with-apr=/opt/rt3/build/apr-1.2.8
make

Which gave no errors, but I think I've tried that before.

rebuilt apache again

with this configure: 

 ./configure --prefix=/opt/rt3/apache --with-apr-util=/opt/rt3/build/apr-util-1.2.8 --with-apr=/opt/rt3/build/apr-1.2.8 --enable-ssl --enable-info --enable-authnz-ldap --with-ssl=/usr/local/ssl --with-ldap --enable-ldap --with-ldap-sdk=openldap

No errors. 

make

My computer crashed during the make and I lost connection to bay. A make install picked up the unfinished sections and continued on.

Completed without errors.

But when I try to start apache, it fails with this fatal error: 

ld.so.1: httpd: fatal: relocation error: file /opt/rt3/apache/bin/httpd: 
symbol apr_ldap_info: referenced symbol not found

Found this advice on a forum: 

1. First compile apr-util module with "--with-ldap" option"
./configure --prefix=/home/openldap/apr-util/ --with-ldap
--with-ldap-include=/home/openldap/Install/include
--with-ldap-lib=/home/openldap/Install/lib/ --with-apr=/home/openldap/apr

2. check config.status file
@apu_has_ldap@,1,;t t

3. Then compile apache server..

Got this error when trying to build apache: 

libtool: link: cannot find the library `/opt/rt3/build/apr-util/libaprutil-1.la'

Added step 2.5: make apr-util

Then tried again to make apache. Wow, that worked. Perhaps the problem was that I failed to do a make install on apr-util.

But now I have to revisit the fastcgi configuration because I get errors in apache logs. But the web server IS running.

RT_SiteConfig was owned by www instead of daemon. Fixing that and restarting apache cleared the errors.

configure ldap rules

I think this is the right apache configuration to do ldap authorization: 

<VirtualHost *>
        ServerName bay.psmfc.org
        AddHandler fastcgi-script fcgi
        Alias /NoAuth/images/ /opt/rt3/html/NoAuth/images/
        ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/
        DocumentRoot /opt/rt3/share/html
        ServerAdmin webmaster@psmfc.org
        SSLEngine on
        SSLCertificateFile /opt/rt3/apache/conf/server.crt
        SSLCertificateKeyFile /opt/rt3/apache/conf/server.key
        <Location />
                AuthName "PTAGIS Web Portal (LDAP) Accounts"
                AuthType Basic
                AuthLDAPURL ldap://ldap.psmfc.org:389/o=psmfc,c=us?uid
                #AuthzLDAPAuthoritative Off
                #SupportNestedGroups On
                require valid-user
                AddDefaultCharset UTF-8
                SetHandler fastcgi-script
        </Location>
</VirtualHost>

But when I try to visit any RT page, I get this error in the apache log: 

[Fri Dec 29 14:07:19 2006] [error] Internal error: pcfg_openfile() called with NULL filename
[Fri Dec 29 14:07:19 2006] [error] [client 10.1.2.59] (9)Bad file number: Could not open password file: (null)

Found a forum that mentioned there is a bug in the documentation. It doesn't tell you that you need this: 

AuthBasicProvider ldap

So now, my complete working rt.conf in the apache/conf/conf.d looks like this: 

# Request Tracker apache config
# depends on fastcgi

LoadModule fastcgi_module modules/mod_fastcgi.so

SetEnv LD_PRELOAD /usr/local/mysql/lib/mysql/libmysqlclient.so
SetEnv LD_LIBRARY_PATH /usr/local/mysql/lib/mysql:/opt/rt3/apache/modules

#FastCgiIpcDir /tmp

FastCgiServer /opt/rt3/bin/mason_handler.fcgi -initial-env LD_LIBRARY_PATH
<VirtualHost *>
        ServerName bay.psmfc.org
        AddHandler fastcgi-script fcgi
        Alias /NoAuth/images/ /opt/rt3/html/NoAuth/images/
        ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/
        DocumentRoot /opt/rt3/share/html
        ServerAdmin webmaster@psmfc.org
        SSLEngine on
        SSLCertificateFile /opt/rt3/apache/conf/server.crt
        SSLCertificateKeyFile /opt/rt3/apache/conf/server.key
        <Location />
                AuthzLDAPAuthoritative Off
                AuthName "PTAGIS Web Portal (LDAP) Accounts"
                AuthBasicProvider ldap
                AuthType Basic
                AuthLDAPURL ldap://ldap.psmfc.org:389/o=psmfc,c=us?uid
                require valid-user
                AddDefaultCharset UTF-8
                SetHandler fastcgi-script
        </Location>
</VirtualHost>
Retrieved from "http://php.ptagis.org/wiki/index.php/Apache_build_on_bay"
Personal tools