Aarrgghh!!

Back to Programming

Wikipedia Scraper

Explanation

To power the Wikipedia Showdown, I had to write a Wikipedia scraper. I'm hope someone out there might use it to do something legitimate.

Sample Usage

Calling this code:

		


1: <cfinvoke component="pathtocfcs.wikipedia" method="weigh" returnvariable="results_1">
2: 	<cfinvokeargument name="search_term" value="diet coke" />
3: </cfinvoke>
4: 			
5: <cfdump var="#results_1#">

Will return:

CONTENTS	Wikipedia, the free encyclopedia   (Redirected from Diet coke) Jump to: navigation, search This article needs additional citations for verification. Please help improve this article by adding reliable references. Unsourced material may be challenged and removed. (January 2008) Diet Coke Type Diet Cola Manufacturer The Coca-Cola Company Country of origin United States Introduced 1982 Variants Diet Coke Caffeine-Free , Diet Coke with Lemon, Diet Coke with Lime, Diet Raspberry Coke, Diet Black Cherry Vanilla Coke, Diet Coke Sweetened with Splenda, Diet Coke Plus Related products Coca-Cola Coca-Cola C2 Coca-Cola Zero TaB Diet Coke (also known as Diet Coca-Cola, Coca-Cola Light or Coke Light; known as No-Calorie Coca-Cola in Japan; known as Calorie Xero Coca-Cola in South Africa; know as Sifr Kalorièe Coke in Saudi Arabia) is a sugar-free soft drink produced and distributed by The Coca-Cola Company. It was introduced in the United States on Independence Day, 1982 as the first new brand since 1886 to use the Coca-Cola trademark. The product quickly overtook Tab in sales. Diet Coke was sweetened with aspartame after the sweetener became available in the U.S. in 1983; however, to save money, this was originally in a blend with saccharin. After Diet Rite cola advertised its 100 percent use of aspartame, and the manufacturer of NutraSweet (then, G.D. Searle & Company) warned that the NutraSweet trademark would not be made available to a blend of sweeteners, Coca-Cola switched the formula to 100 percent NutraSweet, later switching back and doing without the NutraSweet trademark. Diet Coke from fountain dispensers still contains some saccharin (to extend shelf life).[1] Coca-Cola light logo In other countries, where cyclamates are not banned (as they were in the U.S. and the United Kingdom in 1970), Diet Coke or Coca-Cola Light may be sweetened with a blend containing cyclamates, aspartame, and acesulfame potassium. In 2005, under pressure from retailer Wal-Mart (which was impressed with the popularity of Splenda sweetener), the company released a new formulation called "Diet Coke sweetened with Splenda." Sucralose and acesulfame potassium replace aspartame in this version. Early sales were weaker than anticipated; however, Coca-Cola did little advertising for the brand, investing money and advertising in Coca-Cola Zero instead. The introduction of the Splenda-sweetened version of Diet Coke resulted in complaints to bottlers, with store shelves often holding very little of the normal version of Diet Coke.[citation needed] Diet Coke does not use a modified form of the Coca-Cola recipe, but instead an entirely different formula. The controversial New Coke, introduced in 1985 (almost three years after Diet Coke's introduction), used a version of the Diet Coke recipe that contained high fructose corn syrup and had a slightly different balance of ingredients. In 2004, Coca-Cola introduced Coca-Cola C2, which it claims tastes much closer to Coca-Cola but contains half the carbohydrates. In 2005, the company introduced Coca-Cola Zero, a sugar-free variation of regular Coca-Cola. When Tab was released in 1963, the Coca-Cola Company refused to release a diet soda with the Coca-Cola name, fearing that its flagship brand might suffer. Its rival Pepsi had no such qualms, and after the long-term success of its sugar-free Diet Pepsi (launched in 1964) became clear, Coca-Cola decided to launch a competing sugar-free brand under the Coca-Cola name, which could be marketed more extensively than the more anonymous Tab. Diet Coke and Diet Pepsi have capitalized on the markets of people who require low sugar regimens, such as diabetics, athletes, and people concerned with calorie intake. In the UK, a 330 ml can of Diet Coke contains around 1.3 calories (5 kilojoules) compared to 142 calories (595 kJ) for a regular can of Coca-Cola. Contents 1 Brand portfolio 2 Advertising slogans for U.S.A. 3 Debate over health issues 4 Ingredients 5 See also 6 References 7 External links // [edit] Brand portfolio Name Launched Discontinued Notes Picture Diet Coke 1982 The first version of Coca-Cola without sugar. Diet Coke Caffeine-Free 1983 A caffeine free version of Diet Coke and the first extension of the Diet Coke formula. Diet Cherry Coke/Diet Coke Cherry 1988 Available in USA and United Kingdom. Discontinued in Australia and Israel. Diet Coke with Lemon 2001 Still available in Austria, Belgium, Brazil, Canada, Chile, Bosnia and Herzegovina, Germany, Hong Kong, South Africa, Spain, Israel and the United Kingdom. The version sold in Continental Europe uses the Coca-Cola Light formula and is generally thought to match much better with the lemon flavoring than the Diet Coke formula. Diet Vanilla Coke/Diet Coke Vanilla 2002 Still available in Hong Kong, New Zealand (only 300mL and 600mL), Australia, and Bosnia and Herzegovina. Diet Coke with Lime 2004 Available in the USA, Ireland, Finland, Sweden, Canada and Bosnia and Herzegovina. Diet Raspberry Coke June 1, 2005 2006 Available in New Zealand, and Bosnia and Herzegovina Diet Coke Sweetened with Splenda 2005 Available in the USA, and Bosnia and Herzegovina. Coca-Cola Zero 2005 Diet Coke Black Cherry Vanilla 2006 2007 available in Bosnia and Herzegovina. Coca-Cola Light Sango 2005 Only available in Belgium, France, Luxembourg, and Bosnia and Herzegovina Coca-Cola Cherry Zero 2007 Coca-Cola Vanilla Zero 2007 Diet Coke with Citrus Zest 2007 Available in the Bosnia and Herzegovina, and United Kingdom. Diet Coke Plus/Coca Cola Light plus 2007 Available in many European countries and in the USA. In countries whose English is an official language, the soft drink is called Diet Coke. In most of Europe, including Germany and Italy, the drink is marketed as Coca-Cola Light, but often referred to as Cola. Coca-Cola has tried to discourage the use of this generic term,[citation needed] and commercials refer only to "Coke Light" or "Coca-Cola Light." In France and Italy, it is referred as Coca-Cola Light. In French-speaking Canada it is called Coke Diète. In Puerto Rico and Spanish-speaking United States areas it is called Coke Ligera (a translation of "Coke Light"). In Italy, the name Diet Coke was used until the early 1990s. In Mexico, Central, South America and most of the Caribbean (except for countries whose official language is English) it is called Coca-Cola Light. In many English-influenced non-English markets including Israel, it is called Diet Coca-Cola. That was also the case in Iceland although the name Diet Coke was also used until it was changed recently to Coca-Cola Light. In Japan it has been called No Calorie Coca-Cola since April 2007 In India it is called Diet Coca-cola. After lots of campaign against coca-cola in India, still Coca-Cola is selling good in chennai and other cities. [edit] Advertising slogans for U.S.A. The current Diet Coke logo was adopted in 2007 "The time is right" (1982) The band Devo served as spokespersons for just 1982 "Just for the taste of it!" (1983) "The one of a kind" (1984) "Just for the taste of it!" (1985) "Taste it all!" (1993) "This Is Refreshment" (1994) "Just for the taste of it!" (1995) "You are what you drink" (1997) "Get the taste of it" (2000) "Live Your Life" (2001) "Do what feels good" (2002) "It's a Diet Coke thing" (2004) "Life is how you take it" (2005) "Light it up!" (2006) "Yours" (2007) "Enjoyment" (2007) "What life should be like." (2008) [edit] Debate over health issues Wikinews has related news: Fizzy drinks out of New Zealand schools from 2009 The most commonly distributed version of Diet Coke (and majority of beverages using artificial sweeteners) relies on Aspartame, which has been blamed by some scientists and medical professionals for possibly causing serious illnesses (such as cancer, brain tumors, brain lesions, and lymphoma) when consumed in large quantities[2][3]. One of the chemicals produced by aspartame after ingestion is methanol.[4][5] It should be noted that Coca-Cola has now released Diet Coke sweetened with sucralose (also known as Splenda), although it is not as common. See also soft drink controversy. An article published in The Independent on 27 May 2007 highlights the dangers of sodium benzoate in soft drinks. According to the article, the sodium benzoate combines with the ascorbic acid (vitamin C, E300) in the drink to form benzene, a carcinogen. (See main article: Benzene in soft drinks for more.) Concerning the combination of sodium benzoate and citric acid, The Coca-Cola Company has reformulated its products so that the risk of forming carcinogens inside of soft drinks is eliminated.[citation needed] Also, the sodium benzoate was found to break down mitochondrial DNA in living yeast cells.[6] [edit] Ingredients The ingredients in Diet Coke are listed in order of greatest to least amount: Carbonated Water Caramel Color Aspartame Phosphoric Acid Potassium Benzoate (to protect taste) Natural Flavors Citric Acid Caffeine [edit] See also A Diet Coke geyser during a Diet Coke and Mentos eruption Diet Coke and Mentos Eruption - It is possible to cause a sudden high-pressure release of carbon dioxide by inserting multiple Mentos into a container of Diet Coke. The MythBusters found this is caused because of nucleation as well as certain catalysts in the Diet Coke and Mentos themselves. The phenomenon is described in some detail by the General Chemistry Online! FAQ, which also attributes it to nucleation.[7] Dolce & Gabbana - A non-alcoholic drink made with Diet Coke and Grenadine which tastes like Cherry Coke but has fewer calories. [edit] References ^ [1] Suit Alleges Deceit in Fountain Diet Cola Drinks ^ Study Links Aspartame To Cancer, Lymphoma, Leukemia In Rats Fed Sweetener; Some Dispute Results - CBS News ^ http://www.ehponline.org/members/2005/8711/8711.pdf ^ http://home.howstuffworks.com/question536.htm ^ http://www.factmonster.com/ce6/sci/A0804226.html ^ Caution: Some soft drinks may seriously harm your health from The Independent ^ [2] Why Do Mentos Mints Foam When You Drop Them into Soda Pop? [edit] External links Diet Coke official website v • d • e Varieties of Coca-Cola Coca-Cola · New Coke · Coca-Cola Cherry · Coca-Cola with Lime · Coca-Cola Vanilla · Coca-Cola Citra · Coca-Cola Black Cherry Vanilla · Coca-Cola Blāk · Coca-Cola with Lemon · Coca-Cola Raspberry · Diet Coke/Coke Light · Coca-Cola C2 · Coca-Cola Zero · Coca-Cola Cherry Zero · Coca-Cola Light Sango · Diet Coke Plus Retrieved from "http://en.wikipedia.org/wiki/Diet_Coke"
LENGTH	10827
SEARCHTERM	diet coke
URL	http://en.wikipedia.org/wiki/Diet_Coke

Code

Source

		


1: <cfcomponent>
2: 	<!--- Stolen from cflib.org--->
3: 	<cffunction access="public" name="stripHTML" output="false" returntype="string" hint="Removes HTML from input string.">
4: 		<cfargument name="str" type="string" hint="String to clean." required="yes">
5: 		<cfreturn REReplaceNoCase(str,"<[^>]*>","","ALL")>
6: 	</cffunction>
7: 		
8: 	<cffunction access="public" name="weigh" output="true" returntype="struct">
9: 		<cfargument name="search_term" type="string" required="yes" hint="The search term. ">
10: 		
11: 		<cfset search_url="http://en.wikipedia.org/wiki/Special:Search">
12: 		<cfset return_struct.searchTerm=arguments.search_term>
13: 		
14: 		<cfhttp url="#search_url#" method="post" delimiter="," resolveurl="no">
15: 			<cfhttpparam type="formfield" name="search" value="#arguments.search_term#" />
16: 		</cfhttp>
17: 		
18: 		<cfif findNoCase("Search - Wikipedia, the free encyclopedia", cfhttp.FileContent)>
19: 			<cfset return_struct.contents="There are no records for that search term.">
20: 			<cfset return_struct.length=0>
21: 			<cfset return_struct.url="">
22: 			<cfreturn return_struct>
23: 		</cfif>
24: 		
25: 		<cfset contents=cfhttp.FileContent>
26: 		
27: 		<cfset contents_start=FindNoCase("bodyContent", contents)>
28: 		<cfset contents_end=FindNoCase("catlinks", contents)>
29: 		<cfset contents_len=(contents_end-contents_start)>
30: 		
31: 		<cfset contents=Mid(contents,contents_start, contents_len)>
32: 		<cfset contents=stripHTML(contents)>
33: 		<cfset contents_crap=14+9> 
34: 		
35: 		<cfset contents=Mid(contents, 14, Len(contents) -contents_crap)>
36: 		
37: 		
38: 		<cfset return_struct.contents=contents>
39: 		<cfset return_struct.length=Len(contents)>
40: 		
41: 		<cfset retrieved_location=FindNoCase("Retrieved from",contents)>
42: 		<cfset article_url=Mid(contents, retrieved_location, Len(contents) - retrieved_location)>
43: 		
44: 		<cfset article_url=replace(article_url,"Retrieved from", "", "ALL")>
45: 		<cfset article_url=replace(article_url,"""", "", "ALL")>
46: 		<cfset article_url=trim(article_url)>
47: 		<cfset return_struct.url=article_url>
48: 	
49: 		<cfreturn return_struct>
50: 	
51: 	</cffunction>
52: 
53: 
54: 
55: </cfcomponent>