Starting our program
Thursday, February 24, 2011
10:56 AM
Now we have the schema
DESCRIBE log;
log: (time:long, user:chararray, action:chararray, status:chararray, subnet:chararray)
Next, filter out things we aren't interested in
failures = FILTER log BY action=='login' 
and status=='failure'; 
Then, omit the action and status fields (by projection), because we don't need them anymore.
failure2 = FOREACH failures GENERATE user, subnet; 
Now our schema is
DESCRIBE failure2;
failure2: (user:chararray, subnet:chararray)
General principle: filter as small as possible, as early as possible.