File: libdata_install.txt Title: LibData Installation Author: Paul F. Bramscher brams006@umn.edu Date: March 16, 2004 ============================================================================== TABLE OF CONTENTS ============================================================================== 1.0 Infrastructure 2.0 Architecture 3.0 Installation Procedures 4.0 Troubleshooting ============================================================================== 1.0 INFRASTRUCTURE ============================================================================== Hardware Developed and tested on generic intel-based Linux servers and Sun servers. Other hardware (Macintosh) might be possible with some code modifications. Queries can be complex and frequent to render pages. A 2 GHz+ server with 1 GB RAM is recommended for a modest installation in a production environment. Disk footprint for the PHP code itself is well under 5 MB. Additional disk requirements due to database size depends on your local utilization and data growth rate -- and should be monitored as it grows. However, LibData is strongly normalized and unlikely to ever present a disk space issue. Operating System Developed and tested on both Red Hat Linux 9 and Solaris unix. Other operating systems (BSD, Mac OS X, etc.) might be possible with varying degrees of modification. Extensive modification would likely be required to run LibData on a Windows-based computer with an Apache, PHP, and mySQL installation (NOT recommended). Web Server Apache. It is possible to run LibData on a server without a DNS name, although it must be a routable IP. It is strongly recommended to split LibData into two directories, one serving public interfaces, and the other in an SSL location (refer to installation steps below). Database mySQL 3.x. Note that LibData was initially developed on a version of mySQL without support for transactions, and this substantially affects the atomicity of the SQL code (lacking the rollback feature). Other sites have reported successful installation of LibData on mySQL versions 4.x, with minor modifications to the Perl install script. Also note that the mySQL password() function may create different hashes between mySQL versions 3.x and 4.x, so migrating a fully populated production back-end of LibData from one version to another may require resetting (effectively rehashing) user passwords. LibData has not been developed for 4.x at this stage, so additional modification may be necessary. Programming Language Written exclusively in PHP. Coding is structured/function based for simplicity, using PHP object encapsulation only for security (session-related) purposes. As this is version 1.0, without outside grants or special funding, code has certainly not been optimized. However, it is highly self-documented and meaningful variable names have generally been used throughout. Recommended Browsers Note that no client-side programming was done (other than HTML and modest usage of CSS). Therefore LibData ought to be quite widely viewable by most any browser available. Additionally, there are no frames to contend with. Viewing is best at 1024 x 768, particularly in the authoring/administrative environments, though 800 x 600 is also supported. In keeping with the spirit of open source, we recommend Mozilla which renders HTML cleaner than MSIE (for example, no extra spaces when a FORM tag is within a table cell). LibData was developed with Mozilla specifically in mind. ============================================================================== 2.0 ARCHITECTURE ============================================================================== ------------------- Directory Locations ------------------- The system requires two directories, one serving up the public interfaces (hereafter referred to as "public libdata"), another which contains the administration and authoring side ("administration libdata"). General directory path information and other variables may be customized in a file named global_vars.php. This file must be in either (a) BOTH the libdata administration directory AND public directory or (b) in the Apache-defined default location for server-side inclusions. Check with Apache and/or PHP documentation for this. The "include" subdirectory below the libdata administration directory must always be a subdirectory below administration and cannot currently be renamed or moved without some program modifications. (Although an additional Apache-defined include location for the global_vars.php file may be anywhere). Following is an example arrangement: ----------------- Public interfaces ----------------- /htdocs/www/libdata /images /styles ------------------------- Administration interfaces ------------------------- /htdocs/www-ssl/libdata_admin /docs /images /include /install (but should be moved elsewhere at installation Step #11.) Note that the administration side here is piped through SSL (defined by Apache configuration). It is strongly recommended that this portion be SSL, since authentication and user management is handled by this portion of libdata. At the very least, a self-signed server certificate can be created without cost. If this is not possible, then the public and administration portions will likely need to have separate directory names (for example "libdata" for public and "libdata_admin" for the administrative side). If your site is not configured for SSL, you need it only for LibData, and don't mind staff having to click (one time) that they trust the server certificate, following are instructions for creating a self-signed certificate on Red Hat 9: http://www.redhat.com/docs/manuals/linux/RHL-9-Manual/custom-guide/s1-secureserver-overview-certs.html Another good tool, usable for other Linux distributions (tested on SuSE) is TinyCA: http://tinyca.sm-zone.net/ ============================================================================== 3.0 INSTALLATION PROCEDURES ============================================================================== ------------- Initial Setup ------------- Before proceeding to the installation steps, it is necessary to configure PHP and Apache for the following: (A) Interpretation of both .php and .phtml file extensions by the php engine. Changing this setting may vary depending on your installation. (try /etc/httpd/conf.d/php.conf with a default Red Hat 9 install). (B) "register_globals = On" should be set in your php.ini file (try /etc/php.ini with a default Red Hat 9 install). *** Note that future versions of LibData may be written without this required setting, but that LibData's security mechanism is not compromised by setting this ON. The security technique involves storing a session ID in a client side cookie which must match a server stored session ID. Every page and every HTML form submission requires re-checking that the client and server session ID's match. Assuming they match, an access level is pulled from the server and applied only to the current page. This constant re-checking, and storing the actual access level on the server side, makes LibData among the more secure web mechanisms available. Also, unlike built-in PHP session capability, LibData sessions are tied to the IP address. So passing a hacked cookie or GET/POST method -- even with a valid session ID -- would fail unless it was done from the correct IP address. (C) Also in the php.ini file, make sure that "magic_quotes_gpc = Off". gpc stands for get/post/cookie, and turning quote escaping on will create problems for interaction at various layers between HTML, PHP and mySQL. Things might get double-escaped, or not escaped at all depending on whether the string has made the "round trip" to the client and back, has been stored in mySQL, etc. Many PHP developers recommend turning this setting off, and LibData has been created with this in mind. If it must be turned on, some work will need to be done with textInmySQL() and textSearchmySQL() to prevent mySQL errors in app_controls.php. (D) Apache SSL should be +StdEnvVars for the SSL port. (E) Server side Includes should be allowed. There may be additional tweaking based on your installation, but the previous are minimal required settings. ------------------------------------------- (1) Build the Public LibData html directory ------------------------------------------- LibData comes as two tar.gz files. One is named libdata_pos.tar.gz. This tar contains all of the public HTML and PHP code. The "p" in this package refers to the "public" side of LibData. Extracting the tar will produce a libdata_pos directory which should be moved to a web-servable location on an Apache instance. It's possible to rename this directory to something more meaningful to your installation. Whatever directory name is chosen, changes will need to be made in Step #3 below to configure LibData to recognize it. It's possible to rename the directories after LibData is fully populated with data as well -- the directory names are hardcoded in only two configuration files (refer to Step #3 for further details). Changes are simple and go into effect immediately. So feel free to rename libdata_pos to something more useful. The public and administration portions of LibData should have different directory names. It's possible (but not recommended) to give them the same name if the public portion is in a directory like /www/html/libdata and the administrative resides in /www-ssl/html/libdata. But to avoid confusion, we recommend giving them differing names (as well as locations). File permissions and ownerships should be tweaked by a unix administrator as necessary. --------------------------------------------------- (2) Build the Administration LibData html directory --------------------------------------------------- The administration/staff modules are contained in libdata_aos.tar.gz. The "a" in this package refers to the "administrative" side of LibData. Extracting the tar will produce a libdata_aos directory which should be moved to a web-servable location on an Apache instance as with the previous step -- but it is HIGHLY recommended that this directory be in an SSL-served location, since passwords and other information will be sent. As with the public portion of LibData, it's possible to rename this directory to something more meaningful to your installation. Whatever directory name is chosen, changes will need to be made in Step #3 below to configure LibData to recognize it. File permissions and ownerships should be tweaked by a unix administrator as necessary. ------------------------- (3) Tweak global_vars.php ------------------------- There are initially two instances of this file in your installation, one in libdata administration and one in libdata public. The files are initially identical, and should always remain identical. Note that with PHP it's possible to configure a single, default server-side include location. So it's possible to delete one of these files, and move the other to that default include location. (This requires minor tweaking with Apache and/or PHP configuration files.) But it's recommended that you worry about this later -- first get LibData up and running. Both files should be tweaked with information specific to your installation: server name, LibData "system" name, administrator, e-mail contact information, etc. and installation directory locations. Take note of the existing format -- don't add or subtract trailing backslashes. These files can be tweaked any time, during installation or with a fully-populated production LibData. Changes go into effect immediately, there is no process to start/restart, etc. -------------------------------------------------------------------- (4) Create the LibData databases (libdata, libstats, and libsession) -------------------------------------------------------------------- *** IMPORTANT **************************************************************** Ensure that there are NO existing mysql databases named libdata, libstats or libsession on your server. They will be DESTROYED by the next step. Running the following script will DROP all existing LibData references to ensure a clean install of the databases, a base data set, and mysql users. ****************************************************************************** *** Also note that the install script works only with mySQL 3.x. mySQL 4.x has additional fields in the mysql.user table and so the install script may need minor tweaking. At any rate, mySQL user rights should be managed very carefully and this script is not meant to provide a definitive solution from a security standpoint. Essentially the mySQL user named "libdata" must have, at a minimum, select, insert, update, and delete capability for the libdata and libstats databases. The mySQL user "libsession" must have select, insert, update, and delete rights to the libsession database. Refer also to the next section (#5) in this document. To run the install script (mySQL 3.x) go to the install directory in the libdata administrative directory. Run the script named libload.pl, and follow the instructions given. The script must be run on the server hosting the mySQL daemon, and the mySQL root account is probably the best account for this procedure to avoid access denial issues (though if you have another account capable of dropping/adding databases, making changes to the mysql grants table, and reloading/refreshing them, you may use that account instead.) If you encounter errors with this step, double-check the following: (1) The location of the Perl interpreter might not necessarily be /usr/bin/perl. Please adjust the reference in libload.pl accordingly. Also, make sure that this file is executable (chmod +x libload.pl). (1) The mySQL daemon is initialized (for the first time), up and running. (2) The mySQL account/password you supplied is valid, and may access the mySQL command-line client with permissions required to perform the actions listed above. Keep this script in a secure place, since it contains default mySQL password information. Don't re-run it without backing up all data in libdata, libstats, or libsession that you wish to keep! ---------------------------------- (5) Examine mySQL user permissions ---------------------------------- This section refers to the "users" that PHP will use in interacting with mySQL LibData databases (libdata, libstats, and libsession). These users are not to be confused with authoring staff accounts -- those are maintained from within LibData itself (and not stored here). Use whichever mySQL client you are most comfortable with (command-line, phpMyAdmin, mySQL Control Center, etc.) access the internal "mysql" system database. In the mysql.user table the libload.pl script (previous step above) should have created basic libdata and libsession accounts with the following rights. Following is an example of entering the rows manually: INSERT INTO user VALUES (‘%’, ‘libdata’, password(‘t0ught0gu355’), ‘Y’, ‘Y’, ‘Y’, ‘Y’, ‘Y’, ‘N’, ‘N’, ‘N’,’N’,’N’,’N’,’N’,’N’,’N’); INSERT INTO user VALUES (‘%’, ‘libsession’, password(‘3v3nt0gh3r’), ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’,’N’,’N’,’N’,’N’,’N’,’N’); Changing passwords is trivial, and you ought to do it now. For example: UPDATE user SET password = password('newpass') WHERE user = 'libdata'; UPDATE user SET password = password('anotherone') WHERE user = 'libsession'; Passwords must be 6 characters minimum or you won't be able to login -- LibData's login mechanism will reject even valid passwords under 6 characters in length. Be sure to pick a mixture of numbers and letters, and alternate some upper and lower case. *** Note that making changes to the libdata or libsession usernames or passwords will require corresponding changes in the following files (the username and password must match what exists in the database). Since they appear as plain-text in the PHP scripts below, you'll need to manage unix user access to these files carefully: LibData administration path: include/db_connect.php (general libdata database connection include) include/accessClass.php (determines access rights for current user session) include/sessionClass.php (creates, logs out, and validates existing sessions) LibData public path: db_connect.php (general libdata database connection include) *** Changing the passwords on a scheduled basis is recommended, and should be part of an overall security plan. In fact, they should be changed immediately after installing LibData for the first time so you're not running with default and/or known passwords. Note that this may be tweaked further. If, for example, the PHP scripts will be run on the same server as the mySQL daemon, then the "%" in the first column might be tweaked to include only that DNS/IP address. This prevents the accounts from accessing your mySQL server from any other server. The usernames themselves (libdata and libsession) may also be renamed, so long as changes are made in the files discussed in this section (and the next). But this isn't recommended at this stage. LibData may be fine-tuned later -- and there's little reason to rename them anyway. ----------------------------------------------- (6) Examine the mySQL database/host permissions ----------------------------------------------- Next, the db table should be examined with regard to similar issues. Following is an example of a manual INSERT into them: INSERT INTO db VALUES (‘%’, ‘libdata’, ‘libdata’, ‘Y’, ‘Y’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’); INSERT INTO db VALUES (‘%’, ‘libsession’, ‘libsession’, ‘Y’, ‘Y’, ‘Y’, ‘Y’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’, ‘N’); Again, note that the initial "%" in the first column might be tweaked to limit to your particular hostname. If you've renamed the libdata and libsession users (Step #5 above) you'll need to rename them here as well. ------------------------------------ (7) Configure PHP connection strings ------------------------------------ If you've made any changes to the libdata and libsession mysql user passwords (hopefully you have!) double-check that they are correctly referenced in the following files. You'll also need to change the database server strings in each of the files, with an IP or valid DNS name appropriate to the server hosting your instance of LibData: LibData administration path: include/db_connect.php (general libdata database connection include) include/accessClass.php (determines access rights for current user session) include/sessionClass.php (creates, destroys and validates existing session) LibData public path: db_connect.php (general libdata database connection include) --------------------------------------------------------- (8) Reload and refresh the mySQL grant/permissions tables --------------------------------------------------------- The mySQL grants tables should be refreshed and reloaded at this point. This can be done with the mySQL command-line client by the following: mysqladmin -uroot -p reload mysqladmin -uroot -p refresh After each command, it will prompt for the administrator password. ------------------------------------------------------------------------- (9) The LibData system should now be functional. Login as administrator. ------------------------------------------------------------------------- Go to the appropriate url and login as administrator. For example: https://www.yourlibrary.edu/libdata/login.phtml default username: admin default password: libd4t4 (a) Follow the Manager Functions link on the bottom of the main console. (b) Click "Manage Staff List" toward the bottom of the Manager Function menu. (c) Select the radio button for Administrator, System to edit that account. (d) Change the administrator password as desired. Note that passwords, for security purposes, are never displayed on any of the forms. Also, passwords for any account may be purged to NULL. This automatically renders the account inactive (accounts without passwords may not login). Be sure not to purge the password for the admin account, or you'll need to enter mySQL manually, and execute a query like this: UPDATE libdata.staff SET password = password('n3wpassw0rd') WHERE staff_id = 2; or some such in order to log back into the system as administrator. ------------------------------------- (10) Test mySQL single-quote escaping ------------------------------------- Assuming your LibData configuration is now functional, it's a good idea to test proper escaping of single-quotes. Refer to 3.0 Installation Initial Setup Part C above on why magic_quotes_gpc must be turned Off. On the main authoring console, attempt to enter a resource with a single-quote in the title. For example, Here's a Title. (Also note that it is necessary to select some initial master subject or master information type -- refer to the user manuals on why this functionality was programmed). Pick anything from either drop-down box. Failing to do this will result in a screen why it's (as currently programmed) necessary to pick something. On the resource entry form, click the Save New Resource button. Then update the title field by pulling out the quote, for example: Heres a Title. Click the Update Resource button. Then go back and add the single-quote one more time, Here's a Title. Click Update Resource again. This will test both the insert and update SQL functionality. If LibData reports a SQL error, you'll most likely need to make a tweak here: {libdata admin directory location} include/app_controls.php. Look for the function textInmySQL(). Most all of LibData's mySQL insert/update strings are first routed here. Depending on your mySQL installation, you may need to escape single-quotes with a backslash as follows: \'. Comment out the appropriate pass-through ($outgoing = $incoming) and uncomment the nearby line above which performs this escaping. Add additional filters here as desired, and make sure that the corresponding data entry form (generally stored in include/forms.php) calls this function for each form field to be filtered for input or update. Make the same corresponding change in textSearchmySQL(). This function is virtually identical, but filters text going into (some) SELECT type statements. Both textInmySQL() and textSearchmySQL() might be heavily modified for bullet-proofing, possible security issues, etc. Note that double-quotes typically present a problem on many sites in all but textarea based HTML form fields. Future versions of LibData may resolve this (but there are some limits with using HTML as an interface). One possible mechanism is to convert certain special characters to their HTML hex equivalents with textInmySQL. The next step, then, is to convert all search-entry forms into their hex equivalents as well, so that converted data is searched with user input in the identical syntax. Other variants on these themes use the php functions addslashes() stripslashes(). Hopefully everything will be working and you won't need to go down this path. ------------ (11) Cleanup ------------ Assuming you've successfully installed LibData, this step is important from a security standpoint and should not be neglected. Move the install subdirectory (currently in the administrative directory) out of a web-servable location, and give only root users access to it -- since re-running the install script will DROP a production/populated LibData database. It's also recommended to chmod -x libload.pl to prevent a malicious user (or yourself) from making a grave mistake to a production system... ------------------------------ Tips for further configuration ------------------------------ (1) The public libdata db_connect.php may, alternatively, use a different username/password from the administrative db_connect.php file. Furthermore, the public connection need have only SELECT rights to the libdata database, and SELECT and INSERT rights to the libstats database. There are several strategies to fine-tune security, some of them are related uniquely to your institution and levels of paranoia. (2) The LibData db_connect.php file, independently in both the administrative and public halves of LibData contains a function named mysql_tryquery(). This is a light wrapper around the built-in mysql_query() function. Note that there is a variable for debug mode ($db_debug). Setting this to 1 (true) will cause all SQL queries which fail to be output to the screen. This is not recommended for production or public side LibData. However, it can be used in conjunction with a mail() type function to the mail the failed query, mySQL error message, and date/time transparently to the system administrator. This functionality is not supplied (and requires a functioning SMTP gateway). However, the programming to enable this is quite minimal given the centralized error-trapping with mysql_tryquery() in the db_connect.php files. (Remember that administrative and public LibData utilize their own separate db_connect.php files.) ============================================================================== 4.0 TROUBLESHOOTING ============================================================================== Debugging dynamic database applications offers challenges on several fronts. If there are problems using LibData at this point, there are four main areas to debug: (1) Unix. Are the libdata files set to the appropriate permissions? Are they web servable? (2) Apache. Is it serving up other web basic HTML pages? (3) PHP. Create basic "Hello World" PHP programs with both a .php and .phtml extension. Are they being served? Make sure that Apache is configured to interpret both .PHP and .PHTML pages through the PHP parser. Make sure that the global_vars.php files, in both the public and administration locations, have the correct information. It sometimes helps to run the phpinfo() function (visit the http://www.php.net site for documentation on it) for information gathering purposes. Make sure that PHP is configured with register_globals = On (refer to 3.0 Installation Procedures Initial Setup Step A.). To test if this is functioning properly, write a simple .PHTML page with a form POST method to another .PHTML page. Collect the value on the recipient .PHTML page and simply print it for display. If it comes through properly, then a problem with variable passing may be ruled out. (4) mySQL. Make sure that the affected files (db_connect.php on both the public and administration sides, accessClass.php, and sessionClass.php) all have information which corresponds to the name of your mySQL server, and the correct mySQL accounts and passwords. Ensure that those accounts have the appropriate permissions as specified in the installation procedures above. Be sure to refresh and reload the mySQL grant tables also as specified, since some changes to the grants table do not always go into effect immediately. For installation problems or suggestions, please send an e-mail to the developer, Paul Bramscher (brams006@umn.edu), though neither I nor the University of Minnesota can make any guarantees or warranties with regard to LibData or its support. March 16, 2004 Paul F. Bramscher brams006@umn.edu University of Minnesota Libraries