Profile-Based Web-Searching - Essay Example

Profile-Based Web-Searching (Computer Science Project Report) Faculty Table of Contents I. Introduction II. Literature Review III. Software/Tools and Modelling Techniques IV. Conclusion References Appendix I. Introduction Personalized search is the fine-tuning of search results and advertising based on an individual’s preferences, demographic information and other factors. Presumably, the better a search engine understands a user’s interests and preferences, the better able it is to target search results, advertising, sponsored links, etc. [Johnson 2005] Background: Providing profile-based Web searching to users is an emerging concept and was first introduced by Eurekster in 2004. At that time it was biggest surprise that Google, Yahoo and Microsoft (MSN now Windows Live Search) had not introduced this concept initially and a smaller company as compared to Google, Yahoo and Microsoft had introduced this concept of personalized search. In case of personalized search various types of information’s from users are collected and based on that information’s the search content is filtered and displayed to users. Now these days internet (Web) contains lots of information and by searching and specific keywords there is possibility that irrelevant information is returned. Now search engines are working in this regards and comes with the solution of personalized search so as give its users search results based on the user demographic information and interests and taste. In addition these days search history can be also maintained by search engines which users can see on later days. Aim of Study: The study of “Profile-Based Web-Searching” will provide an opportunity to learn how different personalized search engines works and make use of user specific information for assisting web searching process. Research Objectives: The research objectives of the projects are: Study existing methods for user-profiling methods for web searching and layout a design for implementation. Study existing technologies for interface and database design. Implement a preliminary web searching system based on the design. Improve the set of metrics and incorporate the user profile-based method in the web searching system. Refine the user profile-based method, the searching mechanism and the system designed to further improve its effectiveness through comparison with other methods. Hardware/Software Requirements: Perl Hypertext Preprocessor (PHP) server, MySQL RDBMS, development tool (Dreamweaver), MySQL Admin Client interface (Navicat) and Rational Rose. Project Plan: Various phases for Profile Based Web Searching Weeks March April May 1 2 3 4 5 6 7 8 9 10 11 12 Problem Identification Information gathering and Analysis Logical and conceptual design Physical design Physical implementation Testing , Review and Changes Implementation Maintenance Figure 1: Project Plan (Gantt chart) II. Literature Review According to Johnson (2005), personalized search drives financial benefits both in short term and long term. In the short term, advertisers increase revenue by targeting ads and promotions at the users who are most likely to click and buy. In addition, consumers want personalization and are more willing to stay with sites and services that ‘know’ them and respond to their specific lifestyles and preferences. Moreover, sites that employ personalized search automatically filter the Web for consumers based on their individual preferences and saves consumers time and frustration, and bonds users to the site. The techniques used in personalizing Web content are used to personalize Web Search. According to Johnson (2005), there are many ways, but they all require ability that is discussed here. The first thing that a search engine requires is the preferences and demographic information of the users. This can be achieved by search engines in such a way that user’s online experience is not interfered. The next thing is the determination of the type of content and advertising that are the best predictors of relevance for the customers. Now the content that is to be displayed to users are filtered based on the previous information. In the last step these results are presented to user in a way that is easy for the consumers to understand. According to Johnson (2005), the end result of applying personalization to search technology is an enhanced search experience with following considerations. The first thing is here noticed that the sponsored links that are displayed to users should be based on their tastes, interests, and recent search histories. This can increase the likelihood of click through to advertisements and other content. Also one more important thing is taken care here is of advertising ads based on user preferences as well as terms entered by users. The last thing that can be done here is to show Web search results that can be organized by style, meaning, category, etc. These results can be presented under a “My Results” tab so as not to displace unfiltered results. Preliminary Analysis of Some Personalized Search Engines: Eurekster had launched the concept of personalized search engine in 2004. It was a surprise that neither Google nor Yahoo but instead from Eurekster this service launched, which was opens to the general public on 21 January 2004. Before launching this service Eurekster site had been involved in a beta test involving only a few hundred people for couple of months. Eurekster provides many options and some of them are; Search filtered by friends, SearchMates, Sharing searches, sites, and using any search engines in combination with Eurekster [Sullivan 2004]. These days, three big search providers Google, Yahoo and Microsoft dominate personalized search. Google Personalized Search: On March 29, 2004, Google unveiled a new tool that allows people to make their Web searches more personal. This Google Personalized Search was available for testing at the Google Labs page and allowed people to create a profile of their interests, which then influences the Web site links shown when they conduct a search. (Bazeley 2004) On November 2005, Google personalized search has left Google lab and made available to users on 38 domains in addition to google.com. Googles personalized search reorders search results based on history of past searches, giving more weight to topics that interest one. It had also provided a feature to maintain history of searches on Google, allowing revisiting pages previously visited just by scanning history of search. For using this service, one has to sign up for a Google account such as Gmail, AdSense, and other Google services [Sherman 2005]. According to Graham (2005), the new personalized Google home page, which began testing on 19th May, 2005 at labs.google.com, ties together search, personalized news, weather, e-mail, stocks, driving directions and movie listings on the main page. (Bazeley 2005) On 2nd February 2007, Google enhanced its personalized search. Now anyone who signs-up for any Google service using a Google Account (such as Gmail, AdSense, Google Analytics among others) will automatically enroll into three additional Google products: Search History, Personalized Search and Personalized Homepage [Sullivan 2007]. Screenshot 1, 2 and 3 (Appendix) shows the personalized search history of Google personalized Search. Yahoo Personalized Search: In May 2004, Yahoo Company began publicly testing a product called Personal Search that allows users to save and share their search results on Yahoos computers. This new Yahoos Personal Search was focusing on allowing people to save bookmarks in a personal online space and append notes to them. Users can also save search results and share them with others via e-mail, RSS feeds or MyYahoo "modules" that can be sent to other Yahoo users [Bazeley 2004]. At presently Yahoo offers MyWeb 2.0 (Beta) Personalized Search, which is shown in Screenshot 4 and 5 (Appendix). MSN (or Windows Live Search) Personalized Search: Microsoft had recently launched Windows Live Search (previously MSN Search). In which various features that can be personalized according to need of the users is added. Some screenshot of this is shown in Screenshot 6, 7 and 8 (Appendix). Below is the table given in which all the general search and personalized search of Google, Yahoo and Windows Live Search is compared. Search Engine Comparison Chart: Google, Windows Live Search, Yahoo Source: http://www.websearchguide.ca/research/compfram.htm Features Google Search (http://www.google.com) Windows Live Search (MSN) (http://www.live.com/) Yahoo Search (http://search.yahoo.com/) File Types Web (html), pdf, msoffice, mac, corel, swf, rss, xml Web (html), ppt, xls, doc, pdf web, ppt, xls, doc, pdf, xml, rss, rdf, swf Search Levels Simple and Advanced Simple and Advanced Simple and Advanced Boolean Operators +, -, OR, (AND by Default) AND, OR, NOT, +, - +, -, (AND by Default) Truncation/ stemming Automatically stems some words and looks for word variants. No No Restrict by date Advanced Search Advanced: Results Ranking - move slider Advanced - time periods Field Search Advanced: Filters for Title, Format, URL, Domain and site Advanced:   Specify domain, country, and language. Advanced Form: words in title or url. Domain file format Proximity Phrase phrase phrase Sorting Relevance, site Relevance, site,  sliders Relevance, site Cached Page Yes (with date) Yes (with date) Yes Search Aids Spells check Definitions of search terms from Answers.com Recognizes US addresses - map search. US Phonebook search Other number searches Spelling Correction Encarta - Find Facts Spells check Shortcuts - dictionary, encyclopedia, travel, weather. Instant search answers while you type. Personal Search Tools Search History and Personalized Search. Google Notebook for saving clips. Windows Live Favorites - online bookmarks MyWeb2 - save and share pages and bookmarks. Verticals & Tools Search Alerts, Image Search , Video Search (TV, educational, personal), News Search and News Alerts, Newsgroup Search ,Local search , Google Maps for locating places. Scholar - journals and scholarly sites, Books - snippets and some full text, News Archive More through Google Labs and Other Google Services Images Feeds: Search for and subscribe to. Local and Maps - birds eye view on maps for US and UK Live Academic - scholarly Preferences - lists settings for all search services, Images, Video, Audio, Local and Maps, News, Answers – questions, Subscriptions - access for for-fee databases. All Yahoo Services Yahoo Tools for Researchers Toolbar Google Toolbar and Desktop Search Windows Live Toolbar Yahoo Companion Subject Directory Open Directory Project Ranked None Yahoo Directory Sponsored Links Google AdWords Yahoo Marketing and MSN Ad Center Yahoo Marketing Above table shows important feature of all the three most popular search engines. These features had been added to these search engines after various research from a large development team , who all the time try to find new features that users want to see and that can help the users for searching more easily and relevant information from the web. In project “Profile-Based Web-Searching”, initially different common features that are used by all the search engines will be incorporated. After that some of the common features that are used for personalized search such as history, bookmark, etc will be included in “Profile-Based Web-Searching” that will be based on user specific information. The main limitation of the “Profile-Based Web-Searching” is that it is currently based on the existing search engines. In addition, it is college level project and developed by a single developer. There are various methods available based on any software can be developed. Some models are The Waterfall Model, The Agile Model, The RAD Model and The Prototype Model. For this project, Prototype model will be used which is described below: The Prototyping Model: A prototype is a working model that is functionally equivalent to a component of the product. In many instances, the client only has a general view of what is expected from the software product. In such a scenario, where there is an absence of detailed information regarding the input to the system, the processing needs and the output requirements, the prototyping model may be employed. This model reflects an attempt to increase the flexibility of the development process by allowing the client to interact and experiment with a working representation of the product. The developmental process only continues once the client is satisfied with the functioning of the prototype. At that stage, the developer determines the specifications of the client’s real needs. (Pressman 2001) Figure 2: Prototyping Cycle Source (modified): www.cs.pitt.edu/~chang/153/c02process/s32.gif Advantages of Prototyping: Faster development and easier for end users to learn/use. Also development costs less. Fewer changes needed after implementation and development backlog decreases. End-user involvement and users know what to expect at implementation. Because of user/analyst communication requirements easier to determine. Disadvantages of prototyping: Often clients expect that a few minor changes to the prototype will more than suffice their needs. They fail to realize that no consideration was given to the overall quality of the software in the rush to develop the prototype. The developers may lose focus on the real purpose of the prototype and compromise the quality of the product. A prototype will hardly be acceptable in court in the event that the client does not agree that the developer has discharged his/her obligations. According to Powel (2006), there are various methodologies available for designing database models. Each of these different approaches consists of a number of steps. These separate steps are interchangeable, repeatable, and iterative. The following sequence of steps to database model design seems the most sensible and will be used with the prototype model. Requirements analysis: It is the first phase in which information about the nature of the data, features required, and any specialized needs will be covered. Conceptual design: This part will be done with the help of graphical tools (for example Rational Rose.) for making Entity Relationship Diagrams (ERDs). This will be made based on data flow diagram and use case diagram that is made in analysis part of the project. Logical design: Create database language commands to generate table definitions. Physical design: Adjust database language commands to alter the database model for the underlying physical attributes of tables. Tuning Phase: Tuning phase includes appropriate indexing, further normalization, or even de-normalization and security features. (Powel 2006, p.20) III. Software/Tools and Modelling Techniques Technologies Used: Various technologies, tools and languages that will be used for this project are summarized below: Modelling Tool/Language Unified Modelling Language (UML) Rational Rose Front End Technology: PHP Development Tool/Editor: Macromedia DreamWeaver Server Side Scripting Language: PHP Client Side Scripting Language: JavaScript Database Management RDBMS: MySQL Front End for MYSQL: Navicat 2005 or Latest Server Internet Server: PHP Environment: Windows NT UML: The Unified Modelling Language (UML) is the standard modelling language for software and systems development. The underlying premise of UML is that no one diagram can capture the different elements of a system in its entirety. Hence, UML is made up of nine diagrams that can be used to model a system at different points of time in the software life cycle of a system. These diagrams fall in various categories, mainly static, dynamic and Implementation and are given below. Static: Use Case and Class diagram. Dynamic: Object, State, Activity, Sequence and Collaboration diagram. Implementation: Component and Deployment diagram. (Chitnis, Tiwari, & Ananthamurthy 2006) Philippe Kruchten’s 4+1 view model breaks down a model into a set of views, each capturing a specific aspect of system and is shown below. Figure 3: Philippe Kruchtens 4+1 view model Source: (Hamilton & Miles 2006, p. 30) Logical View: A logical view describes the abstract descriptions of a system’s parts and used to model what a system is made up of and how the parts interact with each other. The diagrams provided by UML in this view are class, object, state machine, and interaction diagrams. Process View: A process view describes the processes within the system and is helpful in visualizing what must happen within the system. This view contains activity diagrams. Development View: A development views describes how system’s parts are organized into modules and components and is very useful to manage layers within the systems architecture. This view contains package and component diagrams. Physical View: A physical view describes how the systems design, as described in the three previous views, is brought to life as a set of real-world entities. This view contains deployment diagrams. Use Case View: A use case view describes the functionality of the system being modelled from the perspective of the outside world and is needed to describe what the system is supposed to do. All of the four earlier views rely on the use case view to guide them that is why the model is called 4+1. This view contains use case diagrams, descriptions, and overview diagrams. (Hamilton & Miles 2006, p. 30) Rational Rose: Rational Rose1 is a visual modelling tool for UML. It comes in different versions suited to different requirements. Rational Rose also supports the designing of data models within the same environment. An interesting feature of Rational Rose is the ability to publish the UML diagrams as a set of Web pages and images. This enables user to share and distribute their application design where the Rational Rose tool is not installed. (Chitnis, Tiwari, & Ananthamurthy 2006). Perl Hypertext Preprocessor (PHP): PHP is a programming language designed for producing dynamic web pages. It is used as server side scripting languages along with JavaScript as client side scripting language. It is free software released by Free Software foundation. PHP can be used for three types of purpose server-side scripting, command-line scripting and client side GUI applications. It supports the object-oriented programming and PHP syntax is similar to Perl language. It has large number of free and open source libraries with the core build. The current PHP server version is PHP 5.2, which was released on November 2, 2006. It is mostly used with MySQL RDBMS. MySQL: MySQL database considered as the world most popular open source database. It is capable product as the entire Google search engine is built upon MySQL technology. MySQL supports almost all the functionality of a commercial RDBMS. MySQL ensures that transactions comply with the ACID Model, allows the building of indexes, supports standard data types, and allows for database replication, among other features. MySQL meshes perfectly with the Perl Hypertext Preprocessor (PHP) dynamic web development language. MySQL is released as open source software under the GNU Public License (GPL) and one can download, install and work with MySQL without paying a cent. [Chapple n.d.] Dreamweaver: Dreamweaver is a web development tool that enables users to efficiently design, develop and maintain standards-based website and applications. It supports various languages and technologies such as ASP, ASP.NET, PHP, JSP, CSS, HTML, XHTML and XML. It provides option at the time of development in design view, code view and both by splitting window. Navicat (Database administration MySQL Client): Navicat is client interface for MySQL database. Vital components for the database administrator are control, recovery, and performance measurement. Navicat affords all of these responsibilities through a single interface. It can convert DBF, CSV, text, or XML files to MySQL databases, eliminating time-consuming data entry and the errors that accompany it. MySQL admin client useful features include back-up/restore databases, data transfer, import/export wizard, visual query builder, batch job scheduling, data synchronization, and SSH tunnel. The primary application window allows the configuration of and access to multiple connections to local and remote databases in Windows Explorer style, cascading open to databases, then tables, and ultimately data. IV. Conclusion Personalized search drives financial benefits both in short term and long term. These days many companies such as Yahoo, Google and Microsoft are involved in giving its user personalized search option, in which user can check history, bookmarks for various searches made. This project is also intended to provide to make use of user specific information to assist the web searching process. In this paper, various existing personalized search engines are covered and various technology and tools that can be used for development of such a system is discussed. References: James Pitkow, Hinrich Schutze, Todd Cass, Rob Cooley, Don Turnbull, Andy Edmonds, Eytan Adar, Thomas Breuel, “Personalized search,” Communications of the ACM, 45(9), September 2002. Rasmus Lerdorf, Peter MacIntyre & Kevin Tatroe (2006). Programming PHP. 2nd Edition, O’Reilly: United Sates of America. Tim Converse, Joyce Park & Clark Morgan (2004). PHP5 and MySQL Bible. Wiley Publishing, Inc.: United Sates of America Vikram Vaswani (2005). How to Do Everything with PHP & MySQL. McGraw Hill/Osborne: New York Elizabeth Naramore, Jason Gerner, Yann Le Scouarnec, Jermy Stolz & Michael K. Glass (2005) . Beginning PHP5, Apache, and MySQL ®Web Development. Wiley Publishing, Inc.: United Sates of America Pressman, R.S. 2001. Software Engineering: A practitioner’s Approach, McGraw Hill, New York. P. 23-47. Booch, G., Rumbaugh, J. & Jacobson, I. 2001. The Unified Modeling Language User Guide, Addison Wesley, New Delhi. P. 3-33 Hamilton, K. & Miles, R. 2006. Learning UML 2.0, OReilly Media, Inc, USA. p. 14-36 Powel, G. 2006. Beginning Database Design, Wiley Publishing, Inc., United States of America. p20. Steve Johnson (2005). Personalized Search, January 27, 2005. Retrieved on 21 February 2007 from http://www.imediaconnection.com/content/4977.asp Michael Bazeley (2004). Google to Enable Personalized Searches, San Jose Mercury News (CA); 03/30/2004 Danny Sullivan (2004). Eurekster Launches Personalized Social Search, January 21, 2004. Retrieved on 21 February 2007 from http://searchenginewatch.com/showPage.html?page=3301481 Chris Sherman (2005). Google Personalized Search Leaves Google Labs, November 10, 2005. Retrieved on 21 February 2007 from http://searchenginewatch.com/showPage.html?page=3563036 Danny Sullivan (2007). Google Ramps Up Personalized Search, February 2, 2007. Retrieved on 21 February 2007 from http://searchengineland.com/070202-224617.php Jefferson Graham (2005). Google gets personal, USA Today; 05/20/2005, Section: Money, Pg. 06b Michael Bazeley (2005). Google lets users personalize home page, San Jose Mercury News (CA); 05/20/2005 Michael Bazeley (2004). Yahoo delivers personal search technology, San Jose Mercury News (CA); 10/05/2004 Chitnis, M., Tiwari, P. & Ananthamurthy, L. 2006. UML Tools. Retrieved on January 1, 2007 from http://www.developer.com/design/article.php/1593811 Chitnis, M., Tiwari, P. & Ananthamurthy, L. 2006. UML Overview. Retrieved on January 1, 2007 from http://www.developer.com/design/article.php/1553851 Mike Chapple (n.d.). Introduction to MySQL. Retrieved on 21 February 2007 from http://databases.about.com/od/shareware/a/mysql.htm http://www.apple.com/downloads/macosx/development_tools/navicatdatabaseadministrationmysqlclient.html accessed on 21 February 2007 http://www.adobe.com/products/dreamweaver/ accessed on 21 February 2007 http://www.websearchguide.ca/research/searcomp2.htm accessed on 9 March 2007 http://searchenginewatch.com/showPage.html?page=2155981 accessed on 9 March 2007 http://valenciacc.edu/lrcwest/searchchart.html accessed on 9 March 2007 http://www.searchengineshowdown.com/features/ accessed on 9 March 2007 Appendix: Some Screenshots of Existing Personalized Search Engines Screenshot 1: Creating an Google Account Screenshot 2: Sign-in to Google Account Screenshot 3: Google Search History Page Screenshot 4: Sign-in to Yahoo Search (My Web) Account Screenshot 5: Yahoo Search (My Web) Page Screenshot 6: Windows Live (MSN) Search Page Sign-in Screenshot 7: Windows Live (MSN) Search Page1 Screenshot 8: Windows Live (MSN) Search Page 2 Read More