Troubleshooters.Com Presents

July 1999 Troubleshooting Professional Magazine: Troubleshooting CGI

Copyright (C) 1999 by Steve Litt. All rights reserved. Materials from guest authors copyrighted by them and licensed for perpetual use to Troubleshooting Professional Magazine. All rights reserved to the copyright holder, except for items specifically marked otherwise (certain free software source code, GNU/GPL, etc.). All material herein provided "As-Is". User assumes all risk and responsibility for any outcome.

<--Security Considerations   |  Contents   |   Using Semantic Nets to Model Troubleshooting's Knowledge, part 1-->

The CGI Troubleshooting Toolkit

By Steve Litt
What we'll do in this article is "pull ourselves up by the bootstraps", creating a no-telnet-needed toolkit that works on our ISP. The very most basic and ubiquitous CGI language is the sh shell, so we'll start by getting a shellscript CGI script to say "hello World". Then we'll build a shellscript CGI script to give us the locations of sh, bash, perl and python. From there we'll switch to Perl for the remainder of the toolkit, since Perl is a better CGI environment.

Note that these tools are NOT intended to be a replacement for telnet, even though one could be built. Any web based telnet replacement would be a severe security risk, possibly worse than telnet itself. These tools are intended to confine their activities to a safe "sandbox".  In no case can the user ever type in a command name -- only arguments.

We do not give equivalent Python scripts. We stress Perl because it's more ubiquitous, *not* because it's better or worse than Perl for CGI.

All the scripts in this article are intended to be run as URL's from a browser, and should yield highly informative troubleshooting information in the browser window.

A Good FTP Client

Changing file attributes is beyond the scope of CGI scripts, or at least for safety's sake it should be. Every CGI programmer needs to chmod files and directories, and without telnet the way to do it is with ftp. Your ftp MUST have chmod ability. If you use Linux's built in ftp client you have it. If you ftp from Windows I can recommend WS_FTP. I've also heard good things about LeechFTP. Without an ftp client to do chmod, you cannot do your work. Once you have it, the rest of your CGI debugging work can be done with the tools provided in this article.

hello_sh.cgi: Prove that CGI runs in the directory.

We need the ability to access the shell via CGI. To do so we give the smallest possible shellscript CGI script, called hello_sh.cgi:
  
#!/bin/sh
# PUBLIC DOMAIN, NO WARRANTEE. USE AT YOUR OWN RISK
echo  "Content-type: text/html"
echo
echo "<html><body>"
echo "Hello World<p>"
echo "</html></body>"

Be sure to give it executable and readable access to all, and make sure it's in a directory with executable enabled, and make sure it's in a directory your ISP allows for CGI. When you bring the url corresponding to this file (i.e. http://www.myisp.net/~myname/hello_sh.cgi), it should write the words "Hello World" on the browser. If not, troubleshoot. Take your troubleshooting back to your Linux box if necessary.

Record all steps necessary to get it to work. Did you need to change the #!/bin/sh line? You may need to ask your ISP for the sh shell's location. If they won't give it to you, consider a different ISP.

Did you need a special file or directory permission? The more time you spend recording what needs to be done, the more time you'll save on the next step. Read on...

server_info.cgi: Discover server basics

The next step is to find out where the ISP has placed the tools you'll need. Hopefully, these tools are on the $PATH. If not, we'll need the heavy artillery in the next section. But assuming they are on the path, the following where.cgi will flush them out.
  
#!/bin/sh
# PUBLIC DOMAIN, NO WARRANTEE. USE AT YOUR OWN RISK
echo  "Content-type: text/html"
echo
echo "<html><body><b>"
echo "which perl: `which perl`<p>"
echo "which python: `which python`<p>"
echo "which bash: `which bash`<p>"
echo "which sh: `which sh`<p>"
echo "whoami: `whoami`<p>"
echo "pwd: `pwd`<p>"
echo "echo \$PATH: `echo $PATH`<p>"
echo "</b></html></body>"

Note that the which commands are surrounded by backticks (`). The backtick basically says "pipe this command's output to stdout in the present process, which of course places the output in the browser window.

Troubleshoot as necessary. Once this script works, you'll know the proper first lines of your Perl and/or Python scripts. If you do not find Perl or Python this way, don't give up. It's possible they exist but not on the path. Read on...

locate_special.cgi

Note: DO NOT USE THIS SCRIPT UNLESS server_info.cgi did not yield the location of a tool you absolutely need. The locate_special.cgi script outputs huge bandwidth, which isn't fair to your ISP if used often. If you choose to use this script, be sure to place it where the curious are not likely to find and use it.

  
#!/bin/sh
# PUBLIC DOMAIN, NO WARRANTEE. USE AT YOUR OWN RISK
echo  "Content-type: text/html"
echo
echo "<html><body>"

echo "<H1>locate perl</H1><pre>`locate perl`</pre><br>"
echo "<H1>locate python</H1><pre>`locate python`</pre><br>"
echo "<H1>locate bash</H1><pre>`locate bash`</pre><br>"

echo "</html></body>"

Once again, please run this script ONLY if you have no other way of locating Python or Perl. This script wastes huge bandwidth, and frequent use is not fair to your ISP (and could get you nailed with excess bandwidth charges).

This section concludes shellscript-based CGI scripts, as by now you've managed to locate either Perl or Python. The rest of the tools described in this article are constructed from Perl. I've chosen Perl because it's the most commonly available on servers, but if you need to use Python, the translation should be simple enough.

hello_perl.cgi

This is your proof of concept that Perl CGI works on your computer:
  
#!/usr/bin/perl -w
# PUBLIC DOMAIN, NO WARRANTEE. USE AT YOUR OWN RISK
print "Content-type: text/html\n\n";
print "<html><body>\n";
print "Hello World<p>\n";
print "</html></body>\n";

Once you have this working, you can begin fashioning the rest of your CGI toolkit.

stdin.cgi

This is a test probe into the output of an html form. It shows you the exact stdin coming to the CGI program from the HTML form. You can paste that output into a file and pipe that file into a misbehaving CGI, making your fix/test cycles much quicker. The stdin.cgi file also serves the purpose of showing exactly what a form is outputting. A good troubleshooting test is to take a misbehaving CGI app and replace the form's action with ./stdin.cgi.
  
#!/usr/bin/perl
# PUBLIC DOMAIN, NO WARRANTEE USE AT YOUR OWN RISK
print "Content-type: text/html\n\n";
print "Output of form. Copy next line into clipboard:<P>\n";
while(<STDIN>)
  {
  chomp($_);
  print "$_<br>\n";
  }
print "Copy previous line into clipboard<P>\n";
print "Environment variable CONTENT_LENGTH = " . $ENV{"CONTENT_LENGTH"};

Here is a piece of test html, called test.html, that will test this for you:
  
Content-type: text/html

<html><body>
<p><form action=./stdin.cgi method=post>
              <input name=STRING>
              <input name=Submit type=submit value=SUBMIT></form>
</body></html>

The stdin.cgi program is one of the most valuable tools you have. You'll discover that the form output is different depending on whether you hit Enter or click the Submit button. Before writing CGI to exercise any form output, be sure to hook the form to stdin.cgi to see the various possible inputs to your CGI. Use stdin.cgi early and often.

cgiprobe.cgi

This tiny beauty is the most valuable of the bunch. Inability to see error messages causes 90% of the difficulty debugging CGI. This little script let's us see them. We simply place the name of the script (less the .cgi) in the argument to the call to the &show() subroutine, call cgiprobe.cgi from the browser, and see the error messages on the browser. 
#!/usr/bin/perl -w
# PUBLIC DOMAIN, NO WARRANTEE. USE AT YOUR OWN RISK
use strict;

sub show
  {
  print "Content-type: text/html\n\n";
  print "<html><body>\n";
  print "<pre>\n";
  print `./$_[0].cgi 2>&1`;
  print "</pre>\n";
  print "</html></body>\n";
  }

&show("whatever");  #THIS ARG IS THE ONLY THING YOU CHANGE!!!! 

If you use cgiprobe.cgi to call a CGI program which calls itself, be sure to change the other cgi program so it calls cgiprobe.cgi instead. Take cmdr.cgi (the next tool discussed in this article) as an example. In its printForm subroutine it outputs html source for a form, that form's action being ./cmdr.cgi. If you were to use cgiprobe.cgi to debug cmdr.cgi, you would change that form action to ./cgiprobe.cgi so that subsequent runs of cmdr.cgi would also be debugged.

Security Issues

DO NOT code this script to allow the user to input which CGI script he wants to debug. This would represent a horrible security breach. By hard coding the command to be debugged, we are sure that only the script we authorize is runnable by browser.

cmdr.cgi

This is the Swiss Army Knife of CGI Troubleshooting Tools. You can do all sorts of commands (any commands for which you add buttons). However, extreme caution must be taken to avoid this script causing security breaches.
  
#!/usr/bin/perl -w


# #######################################################################
# cmdr.pl: Web based command runner, version 1.0.0
#   Copyright (C) 1999 by Steve Litt (Steve Litt's email address)
#
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
#
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#
#   You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#   Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
#
# Steve Litt, Steve Litt's email address, http://www.troubleshooters.com
# #######################################################################

use strict;

sub printForm
  {
  my($initVal) = $_[0];
  unless (defined ($initVal)) { $initVal = "";}
  print "<form action=\"./cmdr.cgi\" method=\"post\">\n";
     print "<input type=\"text\" name=\"STRX\" value=\"$initVal\">\n";
     print "<input name=\"DUMX\" type=\"hidden\" value=\"dummy\">\n";

     &makeButtons();

     print "</form>\n";
  }


#########################################################
# List each command in order you want their buttons to
# appear. Commands allowed to have a tail (i.e. ls) should
# end in an elipses, while commands not allowed to have
# a tail (whoami) do not end in elipses.
#########################################################
sub makeButtons
  {
  makeButton("ls -ldF...");
  makeButton("ls...");
  makeButton("which...");
  makeButton("pwd");
  makeButton("whoami");
  }


sub makeButton
  {
  print "<input name=\"BUTX\" type=\"submit\" value=\"$_[0]\">\n";
  }

sub getRawInput
  {
  my($return) = <STDIN>;
  return($return);
  }

sub fixInput
  {
  my($sz) = shift(@_);
  if (not defined($sz))
    {
    $sz = "";
    }
  my $leftangle = shift(@_); unless(defined($leftangle)) {$leftangle = q(&lt;)};
  my $odoa = shift(@_); unless (defined($odoa)) {$odoa = '<br>'};
  my $odoaodoa = shift(@_); unless (defined($odoaodoa)) {$odoaodoa = '<p>'};
  $sz =~ (s/%3C/$leftangle/ge); #special handling for left angle bracket
  $sz =~ (s/%26/&amp/g);     #special handling for & sign
  $sz =~ (s/\+/ /g);         #plus signs sent from form as pluses
  $sz =~ (s/%0D%0A%0D%0A/$odoaodoa/g);
  $sz =~ (s/%0D%0A/$odoa/g);
  $sz =~ (s/%(..)/pack("c",hex($1))/ge);
  return($sz);
  }

sub getTail
  {
  $_[0] =~ /STRX\=(.*?)\&DUMX/;
  return($1);
  }

sub getCmd
  {
  my($cmd);
  if($_[0] =~ m/BUTX\=(.*)$/)
    {
    $cmd = $1;
    if($cmd =~ m/(.*)\.\.\./)
      {
      $cmd = "$1 " . &getTail($_[0]);
      }
    }
 else                #hacker control
    {
    $cmd = "";
    }
  return($cmd);  
  }

sub security_ok
  {
  my($maxdotdots) = 1;  #change to 0 if top dir cgi allowed

  my($cmd) = $_[0];
  my($tail) = $_[1];

  my($return) = 1;

  my(@dotdots) = ($cmd =~ m/\.\./g);
  if($#dotdots + 1 > $maxdotdots)
    {
    print "Only $maxdotdots updirectories allowed.\n";
    $return = 0;
    }
  
  
  if($tail =~ m/^\s\//)
    {
    print "No absolute directories allowed.\n";
    $return = 0;
    }

  if($cmd =~ m/[\|\>\<\;]/)
    {
    print "No [\|\>\<\;] allowed.\n";
    $return = 0;
    }

  my(@wrds) = split(" ", $tail);
  if($#wrds + 1 > 1)
    {
    print "Only one word allowed in command tail.\n";
    $return = 0;
    }
  return($return);
  }

sub main
  {
  print "Content-type: text/html\n\n";
  print "<html><body>\n";

  my($rawstring) = &getRawInput();
  my($fixedstring) = &fixInput($rawstring);
  my($cmd) = &getCmd($fixedstring);
  my($tail) = &getTail($fixedstring);
    
  &printForm($tail);

  print "<pre>$fixedstring</pre><p>\n";
  print "<H1>$cmd</H1><pre><b>\n";

  if (&security_ok($cmd, $tail))
    {
    print `$cmd`;        ### NOTE THE BACKTICKS!!!
    }

  print "</b></pre></body></html>\n";
  }

&main();

You are entirely responsible for the use of this script. Do not disable its security provisions (in security_ok). In spite of the security provisions, cracker exploits are possible. This script should be on the server ONLY when used -- delete it immediately when you're done. And put it in an out of the way place where the curious won't find it.

hellowrite.cgi (writes new file)

CGI that writes files almost always bombs the first few tries. So try this script, which is a proof of concept of file writing in a directory. Since you want to see detailed error messages, be sure to run this script, and any other scripts that write files, from the cgiprobe.cgi tool described earlier in this article.
SECURITY WARNING! Always hard code the name of the file to be written in the in the
my($filetowrite) = 
line. Making this filename any way configurable is a major security breach.
Accuracy warning: Because of situations where there is (or is not) a preexisting copy of the file, you may need to run this cgi script twice to get an accurate assessment of the situation. 
#!/usr/bin/perl -w
# #######################################################################
# hellowrite.cgi: Web based file write tester, version 1.0.0
#   Copyright (C) 1999 by Steve Litt (Steve Litt's email address)
#
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; version 2 of the License.
#
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#
#   THE AUTHORS AND DISTRIBUTORS OF THIS PROGRAM ARE NOT RESPONSIBLE
#   FOR DAMAGE CAUSED BY ITS USE OR MISUSE.
#
#   You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#   Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
#
# Steve Litt, Steve Litt's email address, http://www.troubleshooters.com
# #######################################################################

use strict;

my($filetowrite) = "./test.junk";

sub main
  {
  my($datestring) = "uninitialized";
  my($preexistingfile) = 0;
  print "Content-type: text/html\n\n";
  print "<html><body>\n";
  print "<H1>Erasing old copy of $filetowrite...</H1><pre>\n";

  my($ul) = unlink($filetowrite);
  print "$ul\n";
  if($ul == 0)
    {
    print "Delete not completed: May or may not indicate a problem.\n";
    }
  else
    {
    print "OK.\n";
    }

  print "</pre><H1>Verifying file $filetowrite no longer exists...</H1><pre>\n";
  if(open(JUNK, "<$filetowrite"))
    {
    close(JUNK);
    print "Error: File still exists -- was not deleted.\n";
    print "Try chmod a+x on the directory.\n";
    $preexistingfile = 1;
    }
  else
    {
    print "Presumably OK: cannot read file.\n";
    }

  print "</pre><H1>Writing new copy of $filetowrite...</H1><pre>\n";
  if(open(JUNK, ">$filetowrite"))
    {
    $datestring = localtime();
    print JUNK "$datestring\n";
    print "OK: \"$datestring\" written to file.\n";
    close(JUNK);
    }
  else
    {
    print "Error: could not write file.\n";
    }

  print "</pre><H1>Reading new copy of $filetowrite...</H1><pre>\n";
  if(open(JUNK, "<$filetowrite"))
    {
    my(@lines) = <JUNK>;
    close(JUNK);
    print "\"";
    print @lines;
    print "\" read back from file.\n";
    chomp($lines[0]);
    if($lines[0] eq $datestring)
      {
      print "OK: Data written was read back.\n";
      print "\nSUCCESS: THIS WRITE OPERATION WILL WORK!\n";
      if($preexistingfile == 1)
        {
        print "   B U T :   It might not work without a preexisting file.\n";
        print "   Check directory containing file for chmod a+w.\n";
        }
      }
    else
      {
      print "Error: Data read back does not match data written.\n";
      print "Try chmod a+w on the directory containing the file.\n";
      print "Try chmod a+w on the file.\n";
      }
    close(JUNK);
    }
  else
    {
    print "Error: could not read file.\n";
    }

  print "</body></html>\n";

  }

main();
Steve Litt can be reached at Steve Litt's email address.