# trigram.awk (Yasumasa Someya, March 17, 2001. Based on bigram.awk)
# function: count the numbers of trigrams (any three-word combinations)  in a text, and 
# print out the result in order of decreasing freq.
# usage: jgawk -f trigram.awk INFILE > OUTFILE
{ 
# $0 = tolower($0)                                   # [option] change to lowercase
   gsub(/[.,:;!?"<>\[\]#(){}]/,"")                     # [option] delete .,:;!?"<>[]#(){}
    for(i=1; i<=NF; i++){ 
       trigram = word1 " " word2" " $i         # get trigrams
       word1 = word2                                  # keep track of the previous word
       word2 = $i                                        # ditto
       count[trigram]++                               # count trigram frequencies
      }
       printf "\rCounting trigrams ... %6d lines done.", NR > "CON"
  }
END {for (w in count)
       print count[w], "\t" w | "sortf -rn"         # sorted in decending order (need "sortf")
 }

=======
By replacing the four lines after the "for( )" line with the following, you can change it to a "4-gram" count script. Note that the more the number, the more memory the script requires.

      fourgram = word1 " " word2 " " word3 " " $i 
       word1 =word2 
       word2 = word3 
       word3 = $i 
       count[fourgram]++ 

The AWK version I'm using is jgawk 2.11.1 + 3.0 (Japanaized Gnu Awk for MS-DOS). You also need the "sortf" program , which can be downloaded from: http://download.vector.co.jp/pack/dos/util/text/sort/srtf10at.lzh

=======
The following are the first 10 most frequent trigrams and 4-grams in Alice's Adventure in Wonderland and the Personal Letter Corpus (PCL2000).
 

Alice in Wonderland.		Personal Letter Corpus
(trigrams)	(4-grams)	(trigrams)	(4-grams)
48  the Mock Turtle 29  said the King 28  the March Hare 20  the White Rabbit 20  said the Hatter 19  said to herself 19  said the Mock 18  said the Caterpillar 17  she said to 17  said the Gryphon	19  said the Mock Turtle 16  she said to herself 11  a minute or two   8  said the March Hare   7  said Alice in a   6  in a tone of   6  as well as she   6  in a great hurry   6  well as she could   6  you won't you will	133  Thank you for   58  Would you please   55  look forward to   52  I'd like to   51  let me know   49  Very truly yours   46  I want to   44  a copy of   39  I hope you   35  be able to	64  Thank you for your 30  I look forward to 24  I would like to 20  Please let me know 18  look forward to hearing 18  to hear from you 17  forward to hearing from 17  to hearing from you 17  to let you know 17  Would you please send

Note: If you are interested in bigram statistics, go to Ted Pedersen's website.
 

ID. No.: CPR46
Title: It's a Wonderful Life (Columbia Pictures, 1946)
Directed by: Frank Capra
Screenplay by: Frank Capra
Source: Screenplay Public Domain Database
No. of Words: 17,066

ID. No.: HCK40
Title: REBECCA (Selznick International Pictures, 1940)
Directed by: Alfred Hitchcock
Screenplay by: Robert E. Sherwood;
Source: Screenplay Public Domain Database
No. of Words: 16,062

open_con.awk . . . open contractions.
ad_sp.awk . . . add space after/before punctuation marks.
if#.awk . . .  delete POS tags in comment lines starting with the # mark.
ltr_nbr.awk . . . add ID number to each letter in the corpus. (a modified version of txt_id.awk)

Send me email if you want the above scripts.

ID. No.: Carroll-99
Title: Alice's Adventures in Wonderland (1865) 
Author: Lewis Carroll
No. of words: 26,949

ID. No.: Carroll-101
Title: Through the Looking Glass and What Alice Found There  (1872)
Author: Lewis Carroll
No. of words: 29,888

ID. No.: Twain-27
Title: The Adventures of Tom Sawyer  (1876) 
Author: Mark Twain
No. of words: 65,942

ID. No.: Twain-28
Title: The Adventures of Huckleberry Finn (1884) 
Author: Mark Twain
No. of words: 110,865

# bigram.awk (Yasumasa Someya, July 10, 2000)
# function: count the numbers of bigrams (i.e. any two-word combinations) in a text, and 
# print out the result in order of decreasing freq.
# usage: jgawk -f bigram.awk INFILE > OUTFILE
{ 
# $0 = tolower($0)                            # [option] change to lowercase
  gsub(/[.,:;!?"<>\[\]#(){}]/,"")               # [option] delete .,:;!?"<>[]#(){}
    for(i=1; i<=NF; i++){ 
      bigram = word " " $i                   # get bigrams
      word = $i                                    # keep track of the previous word
      count[bigram]++                         # count trigram frequencies
      }
      printf "\rCounting bigrams ... %6d lines done.", NR > "CON"
      }
END { for (w in count)
      print count[w], "\t"w | "sortf -rn"    # sorted in decending order (=> need "sortf")
}

# "sortf" can be downloaded from: http://download.vector.co.jp/pack/dos/util/text/sort/srtf10at.lzh
# The AWK version I'm using is jgawk 2.11.1 + 3.0 (Japanaized Gnu Awk for MS-DOS).
 

> Suppose you've got the following concordances line (306 lines in total) in response to a search string "[a-z]+_VB [a-z]+_VBG" which matches any sequence of a present-tense verb immediately followed by another verb in the -ing form.

    1  been_BEN able_JJ to_TO absorb_VB rising_VBG costs_NNS by_IN economies_NNS 
  6  N plan._NN I_PRP 'd_MD appreciate_VB knowing_VBG a_ART little_JJ more_JJR
  7         We_PRP would_MD appreciate_VB knowing_VBG as_RB soon_RB as_IN poss
  8  N ,_, we_PRP should_MD appreciate_VB knowing_VBG it_PRP ,_, but_CC if_IF 
  9  NN ,_, and_CC shall_MD appreciate_VB receiving_VBG ,_, at_IN your_PRP$ ea
 ... (omitted)
 41                         Avoid_VB making_VBG personal_NN calls_VBZ unless_I
 42  B that_IN I_PRP may_MD avoid_VB taking_VBG steps_NNS which_WDT neither_RB
 43                   To_TO avoid_VB wasting_VBG your_PRP$ time_NN ,_, may_MD 
 44  ro_NNP Dan_NNP will_MD become_VB acting_VBG manager_NN of_IN the_ART West
 45  NP 1_CD we_PRP will_MD begin_VB assigning_VBG the_ART booth_NN spaces_NNS
 46  d_VBD that_WDT you_PRP begin_VB attending_VBG AA_NNP meetings_NNS each_DT
 47   will_MD have_HV to_TO begin_VB charging_VBG a_ART fee_NN for_IN this_DT 
 ... (omitted)
 91  N ,_, would_MD you_PRP consider_VB making_VBG a_ART donation_NN so_IN we_
 92  pe_VBP you_PRP will_MD consider_VB making_VBG another_DT order_NN with_IN
 93  RP$ family_NN would_MD consider_VB moving_VBG to_TO <_< new_JJ city_NN >_
 94  pe_VBP you_PRP will_MD consider_VB ordering_VBG the_ART 184CZ_CD ,_, whic
...  (omitted)
165  ion._NN you_PRP 'll_MD enjoy_VB working_VBG with_IN him_PRP ._.
166  hat_IN you_PRP will_MD enjoy_VB working_VBG with_IN them_PRP ._.
167  We_PRP shall_MD all_DT enjoy_VB working_VBG with_IN you_PRP ,_, I_PRP 'm_
...  (omitted)
305  vn't_NEG have_HV to_TO work_VB delivering_VBG pizzas_NNS ._.
306  PRP$ dollars_NNS to_TO work_VB earning_VBG the_ART financial_JJ independe

Suppose also that what you want to do from here is to COUNT the numbers of occurrences of respective "VB+VBG" sequences, and, of course, you don't want to do it old way when there are so many to count. 

Here's a Perl script that will do the counting for you. Note that you should first delete all the POS tags (use "del_tag.awk" for this purpose -- see my note of May 2 below) from your text before feeding it to the followind Perl script, so that things will be much easier later. Also, this is a general purpose script and counts the numbers of ANY and ALL two-word combinations, not just the "VB+VBG" sequence.

# bigram.pl (Yasumasa Someya, May 10, 2000)
# function: count the numbers of bigrams (i.e. any two-word combinations) in a text, and 
# print out the result in order of decreasing freq.
# usage: perl bigram.pl INFILE > OUTFILE

while(<>) {
    chop;
    tr/A-Z/a-z/;                                     # [option] change to lower case
    tr/.,:;!?"<>[]#(){}//d;                        # [option] delete punctuation marks and symbols
    foreach $word1 (split) {                 # split line into words
        $bigram = "$word2 $word1";    # get bigrams 
        $word2 = $word1;                     # keep track of the previous word
        $count{$bigram}++;                  # count bigrams  frequencies
    }
}
foreach $bigram (sort numerically keys %count) {     # sort bigrams numerically
   print "$count{$bigram}\t $bigram\n";                     # print "frequency \t trigram \n"
}
sub numerically {                               # compare two words numerically
   $count{$b} <=> $count{$a};            # decreasing order
}
# end of script

The script will return a simple bigram frequency list like the following (only first 10 lines are quoted):

24   appreciate receiving
16   we would
16   to continue
14   we will
14   would appreciate
14   you will
13   to begin
13   would you
10   working with
10   to consider

From this list, you pick up only what you want (Need another scripts? Well, what I recommend for further processing of the data like sorting and counting, etc. is to import the table to MS Excel. Things will be much easier this way), and you get a new list like the following (only first 18 lines are quoted in decending order of frequency):

24   appreciate receiving
9    enjoy working
4    start making
4    continue serving
3    enjoy serving
3    start taking
3    start working
3    begin using
3    begin operating
3    appreciate knowing
3    stop envying
2    consider working
2    continue providing
2    consider recommending
2    stop talking
2    enjoy dealing
2    consider centralizing
2    consider raising

The following is a trigram version of the same script -- in case you want to pick up any three-word combinations:

# trigram.pl (Yasumasa Someya, May 10, 2000)
# function: count the numbers of trigrams (any three-word combinations) in a text, and 
# print out the result in order of decreasing freq.
# usage: perl trigram.pl INFILE > OUTFILE

while(<>) {
    chop;
    tr/A-Z/a-z/;                                                     # [option] change to lower case
    tr/.,:;!?"<>[]#(){}//d;                                         # [option] delete punctuation marks and symbols
    foreach $word1 (split) {                                  # split line into words
        $trigram = "$word3 $word2 $word1";        # get trigrams and count their frequencies
        $word3 = $word2;                                      # keep track of the previous word
        $word2 = $word1;                                      # dotto
        $count{$trigram}++;                                   # count trigram frequencies
    }
}
foreach $trigram (sort numerically keys %count) {     # sort trigrams numerically
   print "$count{$trigram}\t $trigram\n";                     # print "frequency \t trigram \n"
}
sub numerically {                                              # compare words numerically
   $count{$b} <=> $count{$a};                           # decreasing order
}
# end of script

# del_tag.awk (formally "prn_txt.awk"; Yasumasa Someya, May 2, 2000)
# function: delete POS tags from a text tagged in the "word_TAG" format.
# usage: jgawk -f prn_txt.awk INFILE > OUTFILE
{
  gsub("_", " _")           # insert space before the underbar to separate WORD from TAG
  gsub(/ _[A-Z]+/,"")     # delete all the tags heahed by the underbar
  gsub(/( _,| _:| _>| _<| _\.| _\)| _\(| _;| _\?)/,"")    # delete all the symbol tags (e.g. ,_, :_:  >_>, etc.)
  gsub(/ _/,"")               # delete stranded underbar
  gsub(/\$/,"")               # delete stranded $ mark (as in "NNP$")
  printf "\rExtracting text data. Please wait... %7d",NR > "CON" 
  print
}
# end of script

(Sample Input = Concordance Output from BLC POS-tagged Corpus) --------------------------------------------

2   my_PRP$ correspondence_NN ,_, it_PRP is_BE clear_JJ that_IN you_PRP were_BE 
3                                 It_PRP is_BE critical_JJ that_IN you_PRP do_DO 
4   G a_ART moderate_JJ income_NN it_PRP is_BE doubtful_JJ that_IN I_PRP will_MD 
5                                 It_PRP is_BE essential_JJ that_IN a_ART certai
6                                 It_PRP is_BE essential_JJ that_IN we_PRP act_VBP

(Sample Output) -----------------------------------------------------------------------------------------------------------------

2   my correspondence , it is clear that you were 
3                                 It is critical that you do 
4   G a moderate income it is doubtful that I will 
5                                 It is essential that a certai
6                                 It is essential that we act
----------------------------------------------------------------------------------------------------------------------------------------

Note: The original KWIC format will not be maintained in the output, as shown in the above example.