Use WWW::Mechanize to automate resume update on hotjobs.yahoo.com

In this article, we provide an example on how to use WWW::Mechanize to automate your resume update on Yahoo! Hotjobs.

Introduction

There are many web sites that provide job searching service. Yahoo! Hotjobs is one of them. You may have spent a lot of time polishing your resume and it is almost perfect. However, you don't get calls often. Besides, cumulative views of your resume grow very slowly. You don't know the reasons. However, you hear the rumor that recruiters only work on Fridays and they only check resumes that are very recently updated. When you come to know about this, you decide to update your resume as often as possible to attract the attention of those recruiters. However you don't want to do this manually every day. You want to automate this process as much as you can. In this case, WWW::Mechanize can be of your help.

Walk through the Hotjobs web site

You first need to go through the resume update process step by step and note down the details about the links you visit and the forms you fill. You'll need them when you develop your WWW::Mechanize script.

You first need to sign out from Yahoo! if you are already signed in because later on your script needs to automate the sign in process. Then you visit Yahoo! hotjobs as a user who hasn't signed in.

Now you click on the "My Resumes" link to manage your resumes. You don't need to note down the complete link, just some keyword that uniquely identifies the link. For all the links on this page, perhaps this is the only link that contains resume-manager.html.

Then you come to the page that asks you to sign in to Yahoo!. If you have the Web developer Firefox extension installed, you can choose from its toolbar "Forms", then "Display Form Details" to find out that the form has the name "login_form" and that the Yahoo! ID and password fields' names are "login" and "passwd" respectively.

Now you can sign in with your Yahoo! ID and password and you are redirected back to the Resume Manager page. Assuming that you have created one resume, you will find that next to your resume there is a link called "Edit" which is one of the actions you can act on your resume.

Similarly the link contains the word text_editor.html that is unique among all the links in this page. Click on the Edit link and you can modify your resume now. You can find out that the form has the name "mainform" and the title's field name is "resumename". You can just add or remove a space in the title and the click on the "Save and Exit" button to update your resume. The button has the name "submitchanges".

With all the information at hand, you are ready to develop the script.

Develop the script

In your script, you first need to include the following modules.

use WWW::Mechanize;
use Crypt::SSLeay; # required for Yahoo login

Then you can create an object of WWW::Mechanize and have it visit the Hotjobs page.

my $agent = WWW::Mechanize->new(autocheck => 1);
$agent->get('http://hotjobs.yahoo.com/');

Next it should follow the "My Resumes" link.

$agent->follow_link( url_regex => qr/resume-manager.html/i);

Then it should comes to the login page and you need to provide the correct user name and password and submit the login form.

$agent->form_name("login_form");
$agent->field("login", $username);
$agent->field("passwd", $pass);
$agent->submit();

After logging in, you need to follow the redirect link back to Hotjobs.

$agent->follow_link( url_regex => qr/redir/i);

Once you are back to Hotjobs' resume manager page, you can have the agent to following the first edit link.

$agent->follow_link( url_regex => qr/text_editor.html/i, n => 1);

Now you can get the resume's title and add/remove a space as necessary.

$agent->form_name("mainform");
$resumetitle = $agent->field("resumename");
if ($resumetitle =~ /\s+$/) {
  $resumetitle =~ s/\s+$//;
} else {
  $resumetitle .= " ";
}
$agent->field("resumename", $resumetitle);

Then you can have the agent to click the "Save and Exit" button to update your resume.

$agent->click_button(name => "submitchanges", value => "Save and Exit");

Now you are all set. For the complete source code listing, please check the hotjobs.txt file. After downloading it, please rename it to hotjobs.pl.

References

• Kevin Hemenway and Tara Calishain's book "Spidering Hacks": This book covers how to writes scripts to collect information automatically from the Internet. It contains a few recipes on using WWW::Mechanize.

Back to articles on development