Doctor Ray Sison: May 2011

Sunday, May 15, 2011

"Taliban on Twitter for Afghan Internet"

I read the following on Yahoo! News earlier today: http://news.yahoo.com/s/afp/afghanistanunresttechnologyit;_ylt=AqLnRZ3ZyfoisNrMk9s.NHaGOrgF;_ylu=X3oDMTE2bWZsdGExBHBvcwMyBHNlYwN5bl9wcmludHBhZ2UEc2xrA2JhY2t0b3N0b3J5.

These anti-technology extremists sent out their first tweet on May 12. Proof that even they could not ignore "Twitter, the online messaging network that revolutionised global communication." as Usman Sharifi says in this Yahoo! News report.

Saturday, May 14, 2011

"Twitter feed lawsuit underscores power of a tweet"

Interesting article that I just read on Yahoo! Finance:

http://finance.yahoo.com/news/Twitter-feed-lawsuit-apf-126656230.html?x=0

Sunday, May 8, 2011

The OpenCalais API

In a previous post, I described how you could use the document viewer form of OpenCalais) at http://viewer.opencalais.com/) to enter text and use OpenCalais to generate semantic metadata from it. Of course, to create a tool in, say, PHP, it is in order to find some way to do the same thing from a script. From an example on the OpenCalais website, I found the proper syntax to call one of the OpenCalais functions to do so as follows:

http://api.opencalais.com/enlighten/rest?licenseID=YOUR_OPEN_CALAIS_ID&content="Princess Kate is married to Prince William of England."

The two URL arguments are licenseID (I used mine obtained for free from OpenCalais) and the content to be analyzed (my example shows just a small amount of text). What is returned is a daunting amount of RDF data. and below is just a snippet of semantic data that illustrates what OpenCalais generated (including William's nationality):

<rdf:Description rdf:about="http://d.opencalais.com/pershash-1/b4a543bb-34d6-355f-81e5-1f0fb83e1111">

<rdf:type rdf:resource="http://s.opencalais.com/1/type/em/e/Person"/>

<c:name>William</c:name>

<c:persontype>political</c:persontype>

<c:nationality>British</c:nationality>

<c:commonname>William</c:commonname></rdf:Description>

This RDF translates into the following graphical metadata shown below when you enter "Princess Kate is married to Prince William of England." in the text field of the OpenCalais document viewer form:

Also, an online form can be implemented to use this OpenCalais API call, using the following HTML:

<html>

<head>

<title>Calais test page</title>

</head>

<body>

<form action="http://api.opencalais.com/enlighten/rest/" method="post"

accept-charset="utf-8">

licenseID: <input type="text" name="licenseID" />

content: <br />

paramsXML: <br />

</form>

</body>

</html>

The form looks like this:

Again, for the licenseID, use one obtained for free from OpenCalais. For content, provide the data that you want to derive semantic metadata for. For paramsXML, it can be left empty, or filled in with something as described at http://www.opencalais.com/documentation/calais-web-service-api/forming-api-calls/input-parameters).

Twitter Tools and Apps

On April 20, 2011, I found an online Twitter tool called Trendistic (www.trendistic.com). I had been looking at different online Twitter tools and open source apps since getting my Twitter account on April 16, as I thought about ways to process streams of tweets. As seen below, when you enter "Navy SEALs" and click 'Show Trends', you see a dramatic peak of Twitter activity related to Navy SEALs on May 1 upon news of the death of Bin Laden--for obvious reasons!

When I looked at The Twitter Book by Tim O'Reilly and Sarah Milstein, which I bought with some other books on Twitter this spring, I saw several other tools. Going to What the Trend (www.whatthetrend.com), I saw the following trends (#ifyoumarryme being on top!):

This book also mentions another tool that I had seen on my one: Twopular (www.twopular.com), which leverages OpenCalais (that I discuss in a previous blog entry). It, too, shows trending Twitter topics in its own way as shown below:

These three tools do not require you to log in with your Twitter account; nor do they require downloading open source code, as other tools. I am interested in seeing what some of the open source Twitter apps can do...

My First Look into the Twitter API

On April 16, the day that I got my first Twitter account (@RaymondSison), I began to look into how you can grab tweets from a Twitter stream of interest. On Yahoo, I searched on "saving Tweets to XML".

This led me to some interesting info at the following blog entry by Anthony Grace, "Twitter - Saving your Tweets", at
http://www.codersbarn.com/post/2009/12/19/Twitter-Saving-your-Tweets.aspx

The author writes the following (and I followed his instructions by using my first Twitter account @RaymondSison):

Recently, I passed the 700 tweet mark and decided to archive what I had.

But how? I gleaned the following tidbits from "Twitter Tips, Tricks, and Tweets" by Paul McFedries - founder and CEO of Mashable. You can save a local copy of your tweets by entering the following URL in your browser:

http://twitter.com/statuses/user_timeline/account.xml?count=n

where account = your account name, and

n = number of tweets

This will open n number of your tweets to date in XML in your browser.

When I did this with my own Twitter account, using http://twitter.com/statuses/user_timeline/RaymondSison.xml?count=n I got the following XML for my one and only tweet "Hello World" (from Fort Worth, TX)--and this gave me some new insight in how to get raw tweet data that we can further process as desired.

<?xml version="1.0" encoding="UTF-8" ?>

- <statuses type="array">

- <status>

<created_at>Sat Apr 16 22:32:10 +0000 2011</created_at>

<text>Hello World!</text>

<truncated>false</truncated>

<favorited>false</favorited>

<in_reply_to_status_id />

<in_reply_to_user_id />

<in_reply_to_screen_name />

<retweet_count>0</retweet_count>

<retweeted>false</retweeted>

- <user>

<name>Raymond Sison</name>

<screen_name>RaymondSison</screen_name>

<profile_image_url>http://a1.twimg.com/sticky/default_profile_images/default

_profile_4_normal.png</profile_image_url>

<url />

<protected>false</protected>

<followers_count>0</followers_count>

<profile_background_color>C0DEED</profile_background_color>

<profile_text_color>333333</profile_text_color>

<profile_link_color>0084B4</profile_link_color>

<profile_sidebar_fill_color>DDEEF6</profile_sidebar_fill_color>

<profile_sidebar_border_color>C0DEED</profile_sidebar_border_color>

<friends_count>2</friends_count>

<created_at>Sat Apr 16 21:59:05 +0000 2011</created_at>

<favourites_count>0</favourites_count>

<utc_offset />

<time_zone />

<profile_background_image_url>http://a3.twimg.com/a/1302724321/images/themes

/theme1/bg.png</profile_background_image_url>

<profile_background_tile>false</profile_background_tile>

<profile_use_background_image>true</profile_use_background_image>

<geo_enabled>true</geo_enabled>

<verified>false</verified>

<statuses_count>1</statuses_count>

<contributors_enabled>false</contributors_enabled>

<follow_request_sent />

<listed_count>0</listed_count>

<show_all_inline_media>false</show_all_inline_media>

<default_profile>true</default_profile>

<default_profile_image>true</default_profile_image>

<is_translator>false</is_translator>

</user>

<geo />

- <place xmlns:georss="http://www.georss.org/georss">

<name>Fort Worth</name>

<full_name>Fort Worth, TX</full_name>

<place_type>city</place_type>

<url>http://api.twitter.com/1/geo/id/42e46bc3663a4b5f.json</url>

- <bounding_box>

<georss:polygon>32.551584 -97.549053 32.551584 -97.033715 33.047463

-97.033715 33.047463 -97.549053</georss:polygon>

</bounding_box>

<country code="US">United States</country>

</place>

</status>

</statuses>

How I Got Interested in Twitter...

On March 4, 2011, I began to really get interested in learning about Twitter and relating it to my Semantic Web studies. My interest got piqued by the ongoing news associated with the protests and revolts of the Arab Spring (http://en.wikipedia.org/wiki/The_Arab_Spring) and the instrumental use of social networks / media such as Twitter, Facebook, and YouTube by the protesters. Clearly, Twitter is a leading-edge part of our lives right now. Yesterday morning, I heard a radio host quip that one measure of who you are today is how many followers you have on Twitter!

From this inspiration about Twitter, the idea came to mind about interpreting Twitter streams, using the Twitter API to grad tweets and process the raw data as part of an accumulating knowledge / data capture from this social network. In time, this could form the basis for an ontology that could be augmented with data from other ontologies. The data can also be studied for trends, such as frequency of key words.

On April 16, I took the plunge and finally got my first ever Twitter account, @RaymondSison! My first tweet (sent that day) was "Hello World!". On May 1, I sent my second tweet: "The day has come at last: Bin Laden is dead. It's history..."

Saturday, May 7, 2011

OpenCalais for Creating Semantic Metadata

Last fall on October 15, 2010, before even thinking about Twitter, I began looking at OpenCalais (www.opencalais.com and http://en.wikipedia.org/wiki/OpenCalais). When supplied natural language content, Open Calais Web Service automatically processes it and generates semantic metadata--entities, facts and events derived from the "raw" data, such as from a website. It uses technology from ClearForest (www.clearforest.com and http://en.wikipedia.org/wiki/ClearForest).

In other words, with OpenCalais, we have the ability to create semantic meaning of use to us. To illustrate how it works, the OpenCalais site supplies a document viewer at http://viewer.opencalais.com/. When you paste content into the text field and click 'Submit', you get semantic metadata:

Clicking on 'Show RDF' gives us useful RDF XML data that we can further process for whatever we want to do with it (for instance, with XSLT--as described at http://en.wikipedia.org/wiki/XSLT).

Eventually, on October 15, 2010, I downloaded the ClearForest Gnosis add-on (for Internet Explorer) from the OpenCalais site. (After getting a free OpenCalais user account.) This tool automatically applies the same semantic generation as above on a website that you are browsing! I brought up the Gnosis Toolbar (at the left of the browser). And then I applied it to the Wikipedia page for the movie Inception. Notice the classes that were created. And note what comes up when I expanded 'Movie':

You can download Gnosis at http://www.opencalais.com/Gnosis. It is amazing!

Friday, May 6, 2011

Mavericks Win!

While blogging about the Twitter record set by this past week's epochal raid on Bin Laden, I was following Game 3 of the Lakers-Mavericks playoff game. It felt tense as the Lakers stayed ahead of the Mavericks for most of the game--but L.A. never pulled too far away. And Dallas continued their trend of strong fourth-quarter performances to win the game, taking a virtually insurmountable 3-0 lead in this series over the haughty NBA champions of the past two seasons. :)

The resilient Mavericks are now 4 and 0 at home in this year's playoffs; and there was a record crowd in attendance in Dallas to watch the Mavs beat L.A.

Twitter Record Set by Bin Laden Raid

On May 2, 2011, it was widely reported that the news of the successful raid against Bin Laden by our heroic Navy SEALs set records for tweets-per-second and tweets-per-hour. The impact of Twitter continues to astound me. Sajid Farooq of NBC Bay Area reported in "Bin Laden Slays Twitter's Fail Whale" at http://www.nbcbayarea.com/news/local/Bin-Laden-Slays-Twitters-Fail-Whale-121109744.html:

Move over Charlie Sheen, Twitter is winning without you.

News of the death of Osama bin Laden Sunday night sent Twitter into a tailspin that not even the "Fail Whale" could keep up with.

Users of the San Francisco-based micro-blogging site set a record for the most tweets-per-second after President Barack Obama announced the death of America's most wanted.

"Last night saw the highest sustained rate of Tweets ever," Twitter's public relations department tweeted Monday. "From 10:45 - 2:20am ET, there was an average of 3,000 Tweets per second."

The biggest surge of tweets came moments after Obama's announcements.

"At 11 p.m. ET, there were 5,106 Tweets per second," Twitter said in another message. "At 11:45p.m. ET, when Pres. Obama finished his remarks, there were 5,008 (tweets per second)."

Ben Par of Yahoo! News Mashable wrote (at a report whose link is at the bottom of my blog entry):

The death of Osama Bin Laden was one of the most tweeted events in history -- and generated the highest sustained rate of tweets ever. Twitter has released updated statistics on the usage of its platform last night. Previously, the social media company reported that more than 4,000 tweets were sent per second during the beginning and the end of Obama's speech. It now says the real number of tweets was about 25% higher.

At 11:00 p.m. ET, just before Obama's speech, users generated 5,106 tweets per second, the highest single volume of tweets during the night. At 11:45 p.m., just when he finished his speech, Twitter users were sending 5,008 tweets per second.

"Last night saw the highest sustained rate of Tweets ever," Twitter announced in a tweet. "From 10:45 - 2:20am ET, there was an average of 3,000 Tweets per second." That equates to a whopping 27,900,000 tweets in just two hours and 35 minutes.

Between 10:45 p.m. ET and 12:30 a.m. ET, the company says its users averaged 3,440 tweets per second. At its peak, Twitter delivered an average of 12,384,000 tweets per hour.

The new numbers offer a glimpse into just how momentous Osama Bin Laden's demise was. The event surpasses the 3,283 tweets sent per second during Japan's victory over Denmark in the World Cup, and the4,064 tweets sent per second during one of the final moments of this year's Super Bowl.

However, the record for most tweets sent in the same second is still 6,939 -- which happened during the most recent New Year in Japan.

I got this from the following link, and there are many other reports about this Twitter record online:

http://news.yahoo.com/s/mashable/20110502/tc_mashable/bin_ladens_death_sparks_record_124_million_tweets_per_hour_stats.

Doctor Ray Sison