Example: parameterize an attribute of a select
Monday, March 07, 2011
3:00 PM
Suppose we have a relation friends: {name:chararray, friend:chararray}
(Amy,George)
(George,Fred)
(Fred, Anne)
(George,Joe)
(George,Harry)
and want to select people who have a certain 
number of friends. 
We create a parameters file 'params.dat' containing
nfriends 2
and load it via
params = LOAD  'params.dat' USING PigStorage(' ') AS (name:chararray, value:int); 
Then we group the pairs by first friend
groups = GROUP friends BY name;
to get: 
(Amy,{(Amy,George)})
(Fred, {(Fred, Anne)})
(George,{(George,Joe),(George,Harry),(George,Fred)})
and count the number of friends
group2 = FOREACH groups GENERATE group as name, COUNT(friends) as count; 
to get
(Amy, 1L)
(Fred, 1L)
(George, 3L)
Now we need to filter by the number of friends. 
We select the parameter of interest: 
nfriends = FILTER params BY name=='nfriends'; 
nfriend2 = FOREACH nfriends GENERATE value; 
This results in the relation nfriend2: (value: int) containing
(2)
Now we CROSS that relation with the group2 relation 
group3 = CROSS group2, nfriend2;
to get 
(Amy, 1L, 2)
(Fred, 1L, 2) 
(George, 3L, 2) 
And finally, filter by the parameter
group4 = FILTER group3 by group2::count==nfriend2::value;
to get
(George, 3L, 2) 
After which we can strip the parameter from the row: 
group5 = FOREACH group4 GENERATE group2::name AS name, group2::count AS count;
To get the one tuple we want, i.e., 
(George, 3L)  