About these pages | Page building by: |  | Content updated 07-09-2005 07:43am |   |
At the beginning of September 2001, we were alarmed by the number of "odd" entries in the server log of a site we maintain. We had discovered the trail of the Code Red worm. Shortly thereafter we detected Nimda. The following program scans through an Apachie web server log and prepares a report on who and when a worm probe is directed to the site.
/* Code Red log file sniffer -- for Apachie server log file */
/* 9-3-01 BAM */
/* Modified for Nimda detection 9-22-01 BAM */
/* Generalized 9-24-01 BAM */
/* Comments improved 10-25-01 BAM */
/*
Most names are kept symbolic to facilitate adding new worms to the scan or modifying the sort order
A new worm requires the addition of (3) statememnts, the reports automaticaly handle the new worm.
Routine uses an indirect, tag sorted database stored in compound variables.
General variable use:
plague -> Name of worm, modified if necessary, to form a valid Rexx name
WormsT.[j] -> Worm names (external printable text)
WormsR.[j] -> Worm names (internal Rexx name -- for indexing)
IP address specific variables:
SortTag.[m] --> m = index into IPNames [ Ordered list points to IPNames entries ]
SortKey.[n] --> n = index into IPNames [ array of sort keys, built by SortKey() ]
IPNames.[n] -> Sequentially built array of IP addresses
IPData.[n] -> i,j,k ... = index of worm name(s) in Worms.[j]; List of worms detected
IPAccum.plague[n] -> worm specific count of probes
IPTimeIn.plague[n] -> worm specific first probe time stamp
IPTimeOut.plague[n] -> worm specific latest probe time stamp
IPIndex.[IP address] -> n [Reverse look-up of IPName.plague[n], IP addresses modified to form valid Rexx names]
*/
/* Load the function library */
call RxFuncAdd 'SysLoadFuncs', 'RexxUtil', 'SysLoadFuncs'
call SysLoadFuncs
/* Program constants */
InputFile = "access_log" /* Default Input file */
OutputFile = "worms.txt" /* Output file */
/* Worm names (used for display on reports) */
/* When adding new worms, include one line for each worm in this and the following block */
WormsT.0 = 2 /* Count of worms --- when adding worms, update this count */
WormsT.1 = "Code Red" /* Code Red worm */
WormsT.2 = "Nimda" /* Admin-32 worm */
/* Worm names in Rexx variable format (used internally for indexing) -- these names MUST be unique */
WormsR.0 = 2 /* Count of worms --- when adding worms, update this count */
WormsR.1 = "CodeRed" /* Code Red worm */
WormsR.2 = "Nimda" /* Admin-32 worm */
/* Initialize accumulators */
Recs = 0 /* Input records from log file */
IPCounter = 0 /* Index & count of unique IP addresses */
DO i = 1 TO WormsR.0 /* Initialize block of accumulators */
plague = WormsR.i /* Look-up the internal name */
IPCount.plague = 0 /* Initialize worm specific sub total probe count for all IP addresses */
Probes.plague = 0 /* Initialize worm specific grand total probe counter */
END /* do i */
PARSE ARG CommandInput /* Get input file from command line (if any) */
/* Resolve input file */
IF ( CommandInput <> '' ) THEN InputFile = CommandInput /* Input file specified on command line */
/* Resolve full file name and make-up an output file name (output file to be written in the input file directory) */
InputFile = STREAM( InputFile, "C", "QUERY EXISTS" ) /* Check if file exists */
IF ( InputFile = '' ) THEN DO
SAY "Input file: '" || CommandInput || "' does not exist"
EXIT
END
OutputFile = FILESPEC( "D", InputFile ) || FILESPEC( "P", InputFile ) || "worms.txt" /* Output in same directory */
rc = SysFileDelete( OutputFile ) /* Delete the old output file */
rc = LINEOUT( OutputFile, "Worm analysis of:" InputFile STREAM( InputFile, "C", "QUERY DATETIME" ) )
rc = LINEOUT( OutputFile, '' )
/* Loop while there are input records */
DO WHILE ( Lines( InputFile ) > 0 )
Recs = Recs + 1 /* Increment Record counter */
Test = LINEIN( InputFile ) /* Fetch an input line */
/* Scan for and log worms */
/* To expand the search for probes, add lines below -- one for each probe */
SELECT
WHEN ( 0 <> POS("+dir", Test )) THEN RC = LogIPaddress( 2 ) /* Nimda attack */
WHEN ( 0 <> POS("/default.ida", Test )) THEN RC = LogIPaddress( 1 ) /* Code red attack */
OTHERWISE
END /* select */
END /* do */
/* End of input file scan */
/* Report findings */
Say "Logfile Input Records=" Recs
/* Display summary of each worm on console */
DO i = 1 TO WormsT.0 /* Loop through worms */
plague = WormsR.i
Say WormsT.i "Probes=" probes.plague "From" IPCount.plague "IP addresses"
END /* do */
rc = LINEOUT( Outputfile, "Input Records: " || Recs)
/* Write complete report to file */
DO j = 1 TO WormsT.0 /* Loop through worms */
CALL Report j
END /* do */
/* Close files */
rc = LINEOUT( OutPutfile )
rc = LINEOUT( InputFile )
/* Quit */
EXIT
/* ************************************************************ */
/* */
/* Procedures -- Note: Many variables are global */
/* */
/* ************************************************************ */
/* ************************************************************ */
/* */
/* Log server name or IP address */
/* */
/* ************************************************************ */
LogIPaddress:
/* Get arguments: WormIndex = Index in WormsR. */
PARSE ARG WormIndex
plague = WormsR.WormIndex /* Look-up name */
PARSE VAR TEST IPaddress . "[" TS " -" . /* Parse the IP address, Time stamp */
TS = OVERLAY( " ", TS, POS( ":", TS ) ) /* Replace first colon */
HashIP = TRANSLATE( IPaddress, '?!', '/.' ) /* Build IP name */
IF ( LENGTH(HashIP) ) > 200 THEN HashIP = LEFT( HashIP, 200 ) /* Protect from log syntax errors */
IF ( IPIndex.HashIP = 'IPINDEX.' || HashIP ) THEN DO /* Create new entry */
IPCounter = IPCounter +1 /* Increment total count */
IndexP = plague || IPCounter /* Make worm specifc index for this IP address */
IPAccum.IndexP = 1 /* Establish accumulator for this address */
IPTimeIn.IndexP = TS /* Log the worm specific Time Stamp for this IP address */
IPIndex.HashIP = IPCounter /* Make Reverse look-up entry for this IP address */
IPCount.plague = IPCount.plague + 1 /* Increment worm specific probe global sub total */
/* Build new entry's sort key */
TestKey = SortKey( IPAddress ) /* Speed-up */
SortKey.IPCounter = TestKey /* Make a sort Key */
/* Note to new programmers: There are may ways to "sort". The following is a variant of the "bubble sort".
Veteran programmers know that a bubble sort, unless, by accident, the list is already in order, is generally the
slowest possible sort technique, however, it is also the easiest to program. If you are hit by only a few thousand
worm probes (only the worm probes are sorted, not the whole log), don't worry. If you are hit by tens of
thousands of probes, this program will run for a while and you may want to explore other sort methods.
If your list runs into the millions of worms, you may be able to walk to the book store, buy a book on sorts,
study the material, write, debug and run your new program, before this version completes or runs out of RAM.
For small lists the bubble sort will be done before the more elaborate big job sorts are barely off the ground. */
/* Add new IP address to end of list, update the sort tags */
IF ( IPCounter = 1 ) THEN DO /* This is the first worm found */
IPCount.plague = 1 /* Initialize IP addresses specific count of probes */
IPNames.IPCounter = IPaddress /* This is also the first entry for this worm */
SortTag.IPCounter = 1 /* This is also the only Sort tag */
IPData.IPCounter = WormIndex /* Log first worm found for this IP address */
END /* If IPCounter = 1 */
ELSE DO /* Since this is the [n]th entry, we must merge the new IP address in sort order */
TestIP = TRANSLATE( IPaddress ) /* Speed-up */
IPNames.IPCounter = IPaddress /* Add new address at end of IP address list */
Probes.plague = Probes.plague + 1 /* increment global worm specific probe counter */
IPData.IPCounter = WormIndex /* Log first worm found for this IP address */
DO i = IPCounter TO 2 BY -1 /* Scan down the Tags, moving old entries up, as necessary */
Last = i - 1 /* Points to potential merge point of new entry */
TK = SortTag.Last
IF ( SortKey.TK >> TestKey ) THEN DO
SortTag.i = SortTag.Last /* New entry is lower, move this entry up */
END /* do i */
ELSE DO
Last = i /* Mark the internal merge point */
LEAVE /* Exit because merge point is not at bottom of table */
END /* do else */
END /* DO i */
SortTag.Last = IPCounter /* Merge the new entry in Tags array */
END /* else do */
END /* do create new entry */
ELSE DO /* Update data for existing IP address */
HashIP = TRANSLATE( IPaddress, '?!', '/.' ) /* Build IP name */
TP = IPIndex.HashIP /* Look-up index of IP address */
IndexP = plague || TP /* Form Plague & IP address specific index */
IF ( IPAccum.IndexP = "IPACCUM." || IndexP ) THEN DO
IPData.TP = IPData.TP || "," || WormIndex /* Add new worm to list for this IP address */
IPAccum.IndexP = 1 /* Initialize worm specific accumulator */
IPTimeIn.IndexP = TS /* Log time stamp */
END
ELSE IPAccum.IndexP = IPAccum.IndexP + 1 /* Increment count for this worm */
IPTimeOut.IndexP = TS /* Log the Time Stamp for latest probe */
Probes.plague = Probes.plague + 1 /* increment global worm specific probe counter */
END /* do */
/* Remove this comment bracket if you would like to list each attack log entry in the output file.
r = LINEOUT( OutputFile, Test) /* List this attack log entry */
End of removal */
RETURN(1)
/* end of server loging */
/* ************************************************************ */
/* */
/* Write report file for this type of attack */
/* */
/* ************************************************************ */
Report:
/* Get arguments: WormIndex = Index in WormsR. */
PARSE ARG WormIndex
plague = WormsR.WormIndex /* Look-up internal name */
type = WormsT.WormIndex /* Look-up display name */
/* Write report headings */
rc = LINEOUT( OutputFile, '' )
rc = LINEOUT( Outputfile, Type "Probes:" probes.plague "From" IPCount.plague "IP addresses" )
rc = LINEOUT( OutputFile, '' )
rc = LINEOUT( OutputFile, "IP address histogram" )
rc = LINEOUT( OutputFile, '' )
/* Column headings */
rc = LINEOUT( OutputFile, "Hits First Hit Last Hit URL/IP address" )
/* List summary for each IP address */
DO i = 1 to IPCounter
/* Assemble the output line ( assume that report is viewed or printed with fixed pitch font) */
TP = SortTag.i /* Scan Sort tags, look-up the next IP address */
IndexP = plague || TP /* Probe specific index */
Hits = IPAccum.IndexP /* Speed-up */
IF ( Hits <> "IPACCUM." || IndexP ) THEN DO /* Limit to specified worm */
OutLine = RIGHT( ' ' || Hits, 5 ) /* Hits */
PARSE VAR IPTimeIn.IndexP Iday Ihr /* Decode Start Time stamp */
PARSE VAR IPTimeOut.IndexP Oday Ohr /* Decode Stop Time stamp */
OutLine = OutLine || ' ' || LEFT( Iday, 6 ) Ihr /* Build output line */
SELECT /* Minimize clutter, include minimum date & time elements */
WHEN ( Hits = 1) THEN OutLine = OutLine || ' '
WHEN ( Iday = Oday ) THEN OutLine = OutLine || ' ' || Ohr
OTHERWISE OutLine = OutLine || ' ' || LEFT( Oday, 6) Ohr
END /* Select */
OutLine = OutLine || ' ' || IPNames.TP /* IP address */
PP = IPData.TP /* Speed-up */
IF ( POS( ",", PP ) > 0 ) THEN DO /* Check for multiple worms */
OutLine = OutLine || ' See also: ' /* Begining separator */
DO WHILE( PP <> '' ) /* Parse data string and list worms */
PARSE VAR PP RP ',' PP
IF ( RP = WormIndex ) THEN ITERATE /* Skip this worm */
OutLine = OutLine || WormsT.RP || ', ' /* Add worm to list */
END /* DO WHILE */
OutLine = SUBSTR( OutLine, 1, LENGTH( OutLine ) - 2 ) /* Trim extra comma */
END /* do */
rc = LINEOUT( OutputFile, OutLine ) /* Write the line */
END /* Assemble line */
END /* do i */
RETURN
/* ************************************************************ */
/* */
/* Fabricate a sort key */
/* The sort key is a unique character string, built from the current log entry, that ranks this entry
relative to all others -- in the desired order */
/* */
/* This version supports only IP addresses or DNS names. */
/* */
/* ************************************************************ */
SortKey: PROCEDURE
/* Fetch argument */
PARSE ARG IPaddress
/* Decompose & rationalize numeric address (pad with leading blanks if necessary) */
PARSE VAR IPaddress N1 "." N2 "." N3 "." N4
IF ( DATATYPE( N1, "N") & DATATYPE( N2, "N") & DATATYPE( N3, "N") & DATATYPE( N4, "N") ) THEN Test = 1
ELSE Test = 0 /* One or more segments are not numbers */
/* Check potential IP addresses for range */
IF ( (Test = 1) & ( N1<256 ) & ( N2<256 ) & ( N3<256 ) & ( N4<256 )) THEN Key = RIGHT( ' ' || N1, 3 ) || '.' || RIGHT( ' ' || N2, 3 ) || '.',
|| RIGHT( ' ' || N3, 3 ) || '.' || RIGHT( ' ' || N4, 3 ) /* Assemble numeric IP address */
/* Sort URL's by domain */
ELSE DO /* Alpha address key formed by moving domain name to left end of string */
RIPaddress = REVERSE( IPaddress )
PARSE VAR RIPaddress Rcom '.' Rmachine '.' Rtail
com = RIGHT( 'AA' || REVERSE( Rcom ), MAX( LENGTH( Rcom ), 3 ) ) /* Top level domain */
machine = REVERSE( Rmachine ) /* Domain name */
tail = REVERSE( Rtail ) /* Everything else */
Key = com || '.' || machine || '.' tail /* Assemble final key */
END
RETURN( Key )
Rexx Group Charter
Lesson 1 | Lesson 2 | Lesson 3 | Lesson 4
Exercises 1 | Exercises 2 | Exercises 3
Answers to exercises
Exercises 1
Example code
Stock Market | Stock Market_a
About these pages | Page building by: | | Content updated 07-09-2005 07:43am | |