apsquared.net

Web Development and Technology Blog.

Archive for category Web Development

Testing Twitters Location API

Dec 14

Posted by fmapap in Twitter, Web Development | No Comments

While Twitter has released it’s location API it has not fully integrated it into it’s site just yet.  So as a developer if you are annotating tweets from your app with locations it may not be easy to tell if its working.  The simple solution is to look at the RSS feed, for example:  http://twitter.com/statuses/user_timeline/thesuggestr_bot.rss

In the resulting XML if you see (with different values of course) <georss:point>32.898393216 -97.044557447</georss:point> – then you have successfully posted a location.

Remember that the user must opt-in to having the location of his/her tweets shared.

Share This

Tags: ,

Using URLRewrite to rename a website

Apr 29

Posted by fmapap in Java, Lessons Learned, Web Development, theSUGGESTR | No Comments

In the process of renaming our website from MyFriendSuggests.com to theSUGGESTR.com I found that most articles on this refer to using mod_rewrite to accomplish the redirect.  However for most us Java guys this may not be the way to go.  I used URLRewrite to accomplish this name change and have 301(Permanent redirects) from my old URL to my new one. 301’s work best with search engines and will help preserve your page rank.

Here is the URL Rewrite configuration I used:

<rule>
<name>Domain Change</name>
<condition name=”host” operator=”notequal”>www.newdomain.com</condition>
<condition name=”host” operator=”notequal”>localhost</condition>
<from>^/(.*)</from>
<to type=”permanent-redirect”>http://www.newdomain.com/$1</to>
</rule>
<rule>
<name>Domain Change</name>
<condition name=”host” operator=”notequal”>www.newdomain.com</condition>
<condition name=”host” operator=”notequal”>localhost</condition>
<from>^/$</from>
<to type=”permanent-redirect”>http://www.newdomain.com/</to>
</rule>

Note for this to work your old domain must still be pointed at your server (via DNS entry).

301 Redirection Rename Website URLRewrite

Share This

Tags: , , ,

Personalized Restaurant and Bar Recommendations
Introducing MyFriendSuggests.com

Jul 17

Posted by fmapap in MyFriendSuggests, Web Development | No Comments

We are please to announce the official launching of MyFriendSuggests.com a social networking site that provides you personalized suggestions for restaurants, bars, clubs, doctors, grocery stores and just about anything else you can think of.  Unlike other sites where you may find yourself digging through 100’s of reviews from people who may be nothing like you MyFriendSuggests.com provides you a much more personalized experience.

 So how does it work?

  • First you build your friend network.  Add and invite your closest friends, we’ll add their friends, and their friend’s friends (and so on) to your network, sort of like 6 degrees of Kevin Bacon.
  • Then rate your favorite places (restauarants, bars, clubs, doctors, etc).  Rating is as simple as clicking the stars next to a places name.  Also you can add new suggestions by clicking Make Suggestion.  The more places you rate the more accurate our suggestions.
  • Using our proprietary formula we will create suggestions based on your ratings, the ratings of people like you and those in your friend network.  These suggestions will be JUST for YOU, based on YOUR preferences.
  • Also while searching throughout the site you’ll be able to easily see which other users are in your friend network and which users have similar tastes to you. 

We’ve also got some other great features such as:

  • Favorite Neighborhoods allows you to quickly see the new recommendations for your favorite neighborhoods as well as who the experts are for that area.
  • Neighborhood Messages allow you to post a question about a given area.  Maybe your new to town and want advice on a gardner or barbershop. 
  • Friend Messaging allows you to directly message anyone (or all) of the people in your friend network to ask them a question directly.
  • Our Newsletter will send you an email letting you know about new suggestions in your favorite neighborhoods AND any personalized suggestions we come up with using our recommendation system.
  • More great features coming soon!

Since we are a new site we need your help filling up our suggestions and ratings so that we can provide the most accurate recommendations possible.  Register today and invite some friends!

 If you have any thoughts or questions feel free to contact us at info at MyFriendSuggests.com

friend network MyFriendSuggests restaurants social networking suggestions Web 2.0

Share This

Introducing APTags a Java API for Tagging

Jun 16

Posted by fmapap in Java, Web Development | 4 Comments

Tagging pages or items is a big part of the Web2.0 movement.  After doing some mild searching I didn’t find a prebuilt API for working with tags for Java (found some PHP stuff).  So I created one to be used in the next release of our site (www.myfriendsuggests.com).  As we’ve done in the past we decided to make the API available for anyone who wants to use it.  We don’t have a ton of working examples or documentation but the API handles all the DB communication and allows for:

  • Adding tags to items
  • Finding items with a tag (or tags)
  • Finding items tagged by a certain user
  • Generating a tag cloud for a user
  • Generating a tag cloud for an item
  • Other tag queries.

So far this has only been tested with MySQL but if there is interest out there please leave us a comment and we’ll work to verify it for other databases.

 To learn more and download the APTags API click here.

api java tagging tags Web 2.0

Share This

Book Review: Founders At Work

May 13

Posted by fmapap in General Technology, Web Development, Website Marketing | No Comments

I’m not much of a reader, but I just read a book Founders At Work, by Jessica Livingston.  It’s basically a bunch of interviews done with various people who led some of the biggest startups of the past 10-20 years.  I found it to be a real interesting read especially for someone like me, a ‘techie’ who is very interested in the world of startups (especially web startups).   The book gives some great insight into the early days of some of the web’s most successful startups.  It’s real interesting to learn about how many of these sites were started by accident or started with something else in mind and then evolved into the successes they are today.  I recommend this book if you are interested in starting your own business with some friends and colleagues.

Book Review Marketing Startups Website

Share This

Scraping Yahoo! for contacts using JScrape

Apr 29

Posted by fmapap in Java, Scraping, Web Development | 2 Comments

This post builds on my previous post, in which we discuss how to scrape webmail sites for contacts.  Yahoo! is by far the easiest of the sites to scrape (of the major sites).  After you’ve sniffed the URLs used for the login you just need to replace the username and password for the login.  Yahoo! currently does not use any JavaScript tricks or special cookies during the login.  Using JScrape as-is should be sufficient.  The one trick to Yahoo is that it breaks up the address book into seperate pages.  In my solution I dynamically grab these URL’s using the following snippet of code:

public String[] getURLs()
{

 String q = “declare namespace xhtml=\”http://www.w3.org/1999/xhtml\”; \n” +
 ”for $d in //xhtml:ol[@id='abcnav']/xhtml:li/xhtml:a \n”+
 ” return <li> { $d/@href/string() } </li> “;

//pScrape is a com.apsquared.jscrape.PageScraper object that has already logged in to the site.
  List l = pScrape.scrapePageForList(“http://address.yahoo.com/yab/us”, q);
  if (l == null)
    return null;

  String[] ret = new String[l.size()];
  for (int i = 0; i < l.size() ; i++)
  {
    TinyNodeImpl ti = (TinyNodeImpl)l.get(i);
    ret[i] = new String(ti.getStringValue());
  }
  return ret;
}

Note: this may return null if the user account only has a small # of contacts.

For each url returned you need to scrape the page looking for the contacts.  I used the following XQuery for that scrape:

declare namespace xhtml=\”http://www.w3.org/1999/xhtml\”;
for $d in //xhtml:td[@class='contactnumbers']/xhtml:span/xhtml:a
return <li> { data($d) } </li>

That’s about it, as we’ll see in the next few days this is much simpler than many other sites (GMail, Hotmail, AOL) as they require many more tricks to login.

java Programming Scraping Social Marketing Web 2.0 XQuery Yahoo!

Share This

Scraping WebMail sites for contacts using JScrape

Apr 29

Posted by fmapap in Java, Web Development, Website Marketing | 2 Comments

Many new websites, especially those that depend on social networks, are now offering ways to import contacts from various WebMail sites.  I’m not going to go into the ethics of asking a user for their user name and password to a webmail site and scraping the site but I will touch on the technical challenges.  I started by building JScrape, a Java API that makes scraping websites easier.  I then decided to try to scrape contact lists from Yahoo!, GMail, Hotmail and AOL.  I found that each of these sites had their own challenges.  The easiest by far was Yahoo!, so that is what I’ll start with.  I’m not going to provide the exact code but will give you tips that will definetly get you going.

The basic process for all of these sites is:

1) Use a tool (such as Fiddler or Ethereal) to capture the network traffic that occurs when you login to the site.
2) Each site will use different cookies and JS to make logging in more challenging (this is the hard part). 
3) Use the same session and post to the address book page for that site.
4) Use JScrape to parse out the email addresses that you want.  You may need to page through different pages depending on the number of email addresses (and how the site displays the addresses).

Sounds simple eh?  Well step #2 can be quite challengine and frustrating.  I will add a new blog entry for each of the different sites and how to “login” to them, so check back soon. 

java Scraping Social Marketing Technorati Web 2.0 Yahoo!

Share This

Improving performance of Taste using DBCP

Apr 25

Posted by fmapap in Java, MyFriendSuggests, Taste, Web Development | No Comments

For the past few weeks I’ve been playing with Taste, a Java based framework for collaborative filtering (basically the recommendation feature found on sites like Amazon and Netflix).    Hopefully in the near feature this tool will be incorporated in our site, MyFriendSuggests.com to improve our suggestion algorithms. 

What I found was the initial description of using a MySQL DataSource sounded fine, but do to the heavy access to the database performance was bad.  Actually it would stop being able to find new connections since the connections were being grabbed faster than windows was cleaning up open sockets.  Simple solution to this was to use the Apache DBCP for db connection pooling.  All I needed to do was add commons-dbcp and commons-pool to my class path and then create a simple function:

public static DataSource getDataSource()
{
  BasicDataSource md = new BasicDataSource();
  md.setDriverClassName(“com.mysql.jdbc.Driver”); 
  md.setUrl(“jdbc:mysql://localhost:3306/dbname”);
  md.setUsername(“user”);
  md.setPassword(“pass”);
  return md;
}

I call this method in the constructor of the MySQLJDBCDataModel class.  After doing that things started performing much better.

java MyFriendSuggests taste Technorati Web 2.0

Share This

Tableless Website development

Apr 15

Posted by fmapap in Lessons Learned, Web Development | No Comments

This is probably old news to everyone but just in case there are other people like me out there I figured I would make this post. 

When I started building my site, I began hand-coding a layout of HTML tables in my JSP code.  That was definitely a mistake.  Using HTML tables to do your website layout is a tedious and not flexibile when it comes to making changes.  About halfway through the development I started doing research (yea I know research should probably have come before I got 1/2 way done) and read that most people were no longer using table but rather tableless css based development.  Unfortunately since I work on this site and my other site as a part time hobby I didn’t have time to scrap the design (although I probably should have).  So the point of this blog?  If you are someone new starting a website I recommend you follow the tableless site development pattern and throw the html tables out the window.  You can find lots of great info on tableless site development right from google.

css html Website

Share This

Setting up eclipse to run web-app under root context

Apr 10

Posted by fmapap in Eclipse, Web Development | No Comments

After fighting with eclipse a little I was able to get my eclipse 3.2 running with WST to deploy my web application to the root context with Tomcat. 

 The trick was that I had to manually edit the file:

<Workspace>\<project>\.setting\org.eclipse.wst.common.component

I had to remove the value in the context root line so it looks like:

<property name=”context-root” /> 

Then I updated the server.xml for my server and changed the autoDeploy setting to false, restarted eclipse and now I can access my site using just http://localhost:8080/. 

Hope this helps save someone else some time in the future!

technorati programming eclipse wst tomcat

Share This

Close
E-mail It