CSCI E-153, Web Development Using XML

November 16, 2010

Harvard University
Division of Continuing Education
Extension School

Course Web Site: http://cscie153.dce.harvard.edu/

Instructor email: david_heitmeyer@harvard.edu
Course staff email: cscie153@dce.harvard.edu

Announcements and Reminders

Web Services - Definitions

Web Service
a software system designed to support interoperable machine-to-machine interaction over a network
(typically conveyed using HTTP with an XML serialization in conjunction with other Web-related standards)
Service Oriented Architecture (SOA)
An application architecture in which all functions, or services, have invokable interfaces that are called to perform business processes. Each interaction is independent of each and every other interaction and independent of the communication protocols used.
Resource Oriented Architecture (ROA)
ROA is a way of turning a problem into a RESTful web service: an arrangement of URIs, HTTP, and XML that works like the rest of the Web, and that programmers will enjoy using.

Web Services - More Readings

HTTP Transaction

Hypertext Transfer Protocol

Web Services - Features

Web Services Protocol Stack

FunctionTechnologies
XML MessagingXML (REST), SOAP, XML-RPC
  • SOAP, Simple Object Access Protocol
  • XML-RPC, XML Remote Procedure Call
  • REST, Representational State Transfer
DiscoveryUDDI
Universal Description, Discovery, and Integration
DescriptionWSDL
Web Services Description Language
TransportHTTP, SMTP, FTP, BEEP

A Web Services Client in Action

Distance between two places, using the Zip Codes Service

Perl code:

#!/usr/bin/perl -w

# SOAP Lite is Perl Library for handling SOAP
use SOAP::Lite;

# GeoPlaces is a local perl module created from the WSDL
# WSDL at
# http://codebump.com/services/placelookup.asmx?wsdl
use GeoPlaces;

# @ARGV is an array of command-line arguments
my ($place1, $state1, $place2, $state2) = @ARGV;

# Create a new zipcode web service object
my $zipcodews = GeoPlaces->new();

# call the "GeoDistance BetweenPlaces" method
my $distance = $zipcodews->GetDistanceBetweenPlaces($place1,$state1,$place2,$state2);

print "Distance between \n";
print "$place1, $state1 and $place2, $state2 is \n";
print "$distance miles \n";

Output:

$ perl distance.pl Cambridge MA Seattle WA
Distance between 
Cambridge, MA and Seattle, WA is 
2459 miles 
      

Under the Hood

SOAP Request

POST http://codebump.com/services/placelookup.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Accept: application/soap
Content-Length: 1161
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://skats.net/services/GetDistanceBetweenPlaces"

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope 
    xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:tns="http://skats.net/services/">
    <soap:Header>
        <tns:AuthenticationHeader>
            <SessionID xsi:type="xsd:string">UNIQUE_ID</SessionID>
        </tns:AuthenticationHeader>
    </soap:Header>
    <soap:Body>
        <tns:GetDistanceBetweenPlaces>
            <place1 xsi:type="xsd:string">Cambridge</place1>
            <state1 xsi:type="xsd:string">MA</state1>
            <place2 xsi:type="xsd:string">Seattle</place2>
            <state2 xsi:type="xsd:string">WA</state2>
        </tns:GetDistanceBetweenPlaces>
    </soap:Body>
</soap:Envelope>

Response

HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Date: Tue, 17 Nov 2009 17:45:43 GMT
Server: Microsoft-IIS/6.0
Content-Length: 618
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 17 Nov 2009 17:45:45 GMT
Client-Peer: 216.10.251.161:80
Client-Response-Num: 1
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" 
    xmlns:types="http://skats.net/services/" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <soap:Body soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
        <types:GetDistanceBetweenPlacesResponse>
            <GetDistanceBetweenPlacesResult xsi:type="xsd:float">2459</GetDistanceBetweenPlacesResult>
        </types:GetDistanceBetweenPlacesResponse>
    </soap:Body>
</soap:Envelope>

REST Example for Distance Between Places

http://.../GetDistanceBetweenPlaces?AuthenticationHeader=authentication_id&place1=Cambridge&state1=MA&place2=Seattle&state2=WA

ParameterParameter Value
AuthenticationHeaderauthentication_id
place1Cambridge
place2Seattle
state1MA
state2WA
GET http://codebump.com/services/placelookup.asmx/GetDistanceBetweenPlaces?....
Cache-Control: private, max-age=0
Date: Tue, 17 Nov 2009 21:50:11 GMT
Server: Microsoft-IIS/6.0
Content-Length: 110
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 17 Nov 2009 21:50:15 GMT
Client-Peer: 216.10.251.161:80
Client-Response-Num: 1
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET

<?xml version="1.0" encoding="utf-8"?>
<float xmlns="http://skats.net/services/literalTypes">2458.80933</float>

REST: Representational State Transfer


Stock Quotes

SOAP

soap

REST with GET

$ lwp-request -US 'http://.../WebServices/StockQuotes.asmx/GetQuotes?QuoteTicker=BA'
GET http://.../WebServices/StockQuotes.asmx/GetQuotes?QuoteTicker=BA
User-Agent: lwp-request/2.06

GET http://.../WebServices/StockQuotes.asmx/GetQuotes?QuoteTicker=BA --> 200 OK
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfQuote xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://swanandmokashi.com">
  <Quote>
    <CompanyName>BOEING CO</CompanyName>
    <StockTicker>BA</StockTicker>
    <StockQuote>78.15</StockQuote>
    <LastUpdated>8/8/2006 4:02pm</LastUpdated>
    <Change>-1.60</Change>
    <OpenPrice>79.55</OpenPrice>
    <DayHighPrice>80.83</DayHighPrice>
    <DayLowPrice>77.84</DayLowPrice>
    <Volume>5646300</Volume>
    <MarketCap>62.122B</MarketCap>
    <YearRange>62.01 - 89.58</YearRange>
    <ExDividendDate>May 10</ExDividendDate>
    <DividendYield>1.38</DividendYield>
    <DividendPerShare>1.10</DividendPerShare>
</PercentChange>ge>-2.01%
  </Quote>
</ArrayOfQuote>

Some interesting Web Services

Amazon Books

<?xml version="1.0" encoding="UTF-8"?>
<ProductInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://xml.amazon.com/schemas2/dev-lite.xsd">
   <Details url="http://www.amazon.com/gp/product/0321442458%3ftag=webservices-20%26link_code=xm2%26camp=2025%26dev-t=D18YF03A0SIZ5Q">
      <Asin>0321442458</Asin>
      <ProductName>UNIX, Third Edition (Visual QuickStart Guide)</ProductName>
      <Catalog>Book</Catalog>
      <Authors>
         <Author>Deborah S. Ray</Author>
         <Author>Eric J. Ray</Author>
      </Authors>
      <ReleaseDate>02 August, 2006</ReleaseDate>
      <Manufacturer>Peachpit Press</Manufacturer>
      <ImageUrlSmall>http://ec1.images-amazon.com/images/I/01YwS5zKFfL.jpg</ImageUrlSmall>
      <ImageUrlMedium>http://ec1.images-amazon.com/images/I/21rQWY9f7sL.jpg</ImageUrlMedium>
      <ImageUrlLarge>http://ec1.images-amazon.com/images/I/414Nd3Jb7vL.jpg</ImageUrlLarge>
      <ListPrice>$24.99</ListPrice>
      <Availability>This item is currently not available.</Availability>
      <UsedPrice>$10.50</UsedPrice>
   </Details>
</ProductInfo>

Amazon S3

Simple Storage Service

Using GET and POST, you can create 'buckets' and upload files to these buckets. Can access the files with authentication or making them 'public'.

#!/usr/bin/perl
use strict;
use warnings;

use S3::AWSAuthConnection;
use S3::QueryStringAuthGenerator;
use HTTP::Date;
use Data::Dumper;

# you'll need to put in your own keys for this to work
my $AWS_ACCESS_KEY_ID = "xxxx";
my $AWS_SECRET_ACCESS_KEY = "xxxx";

my $conn = S3::AWSAuthConnection->new($AWS_ACCESS_KEY_ID, $AWS_SECRET_ACCESS_KEY);

my $BUCKET_NAME = "dce-xmlwebdev";
my $res = $conn->create_bucket($BUCKET_NAME);
print "Created bucket ",$BUCKET_NAME," ",$res->http_response->message, "\n";

# PUT file
my $img = "ma_purple.png";
# slurp in file
my $data = do { local( @ARGV, $/ ) = $img ; <> } ;
my $s3obj =  S3::S3Object->new($data);
my $key = 'images/'.$img;
$res = $conn->put(
                 $BUCKET_NAME,
                 $key,
                 $s3obj,
                 { "x-amz-acl" => "public-read",
                   "Content-Type" => "image/png" }
                 );

print "uploaded file ",$img,"  ",$res->http_response->message, "\n";

my $list = $conn->list_bucket($BUCKET_NAME)->entries;
print "---- bucket contents ----","\n";
print Dumper $list;

my $generator = S3::QueryStringAuthGenerator->new($AWS_ACCESS_KEY_ID, $AWS_SECRET_ACCESS_KEY, 0);

print "---- URLs to access image ----","\n";
foreach my $calling_format ( qw/REGULAR SUBDOMAIN/ ) {
    $generator->set_calling_format($calling_format);
    my $public_url = $generator->get($BUCKET_NAME, $key);
    print substr($public_url, 0, index($public_url, "?")), "\n";
}

exit;

$ perl map.pl
Created bucket dce-xmlwebdev OK
uploaded file ma_purple.png OK
---- bucket contents ----
$VAR1 = [
          {
            'Owner' => {
                       'ID' => '0690d3b40ed7a7d51399c4437631fff8fe3ef73aefee0c251613cd5ab56434a4',
                       'DisplayName' => 'heitmey'
                     },
            'Size' => '110586',
            'ETag' => '"53527f16b144f143c19b2deecd2761b8"',
            'StorageClass' => 'STANDARD',
            'Key' => 'images/ma_purple.png',
            'LastModified' => '2007-08-07T15:44:07.000Z'
          }
        ];
---- URLs to access image ----
http://s3.amazonaws.com:80/dce-xmlwebdev/images%2Fma_purple.png
http://dce-xmlwebdev.s3.amazonaws.com:80/images%2Fma_purple.png

Alexa Web Information Service (AWIS)

The Alexa Web Information Service makes Alexa's vast repository of information about the traffic and structure of the web available to developers.

URL Info

http://awis.amazonaws.com?
AWSAccessKeyId=zzzzzzzz&
Action=UrlInfo&
ResponseGroup=Rank&
Timestamp=2009-11-17T22%3A41%3A00.000Z&
Signature=QD9709U9FynGh1QpviVcyMnNQMQ%3D&
Url=http://www.harvard.edu/

<aws:UrlInfoResponse xmlns:aws="http://alexa.amazonaws.com/doc/2005-10-05/">
    <aws:Response xmlns:aws="http://awis.amazonaws.com/doc/2005-07-11">
        <aws:OperationRequest>
            <aws:RequestId>bbe8b0b9-6d96-1dad-9e14-7e8822ba8053</aws:RequestId>
        </aws:OperationRequest>
        <aws:UrlInfoResult>
            <aws:Alexa>
                <aws:TrafficData>
                    <aws:DataUrl type="canonical">harvard.edu/</aws:DataUrl>
                    <aws:Rank>1635</aws:Rank>
                </aws:TrafficData>
            </aws:Alexa>
        </aws:UrlInfoResult>
        <aws:ResponseStatus xmlns:aws="http://alexa.amazonaws.com/doc/2005-10-05/">
            <aws:StatusCode>Success</aws:StatusCode>
        </aws:ResponseStatus>
    </aws:Response>
</aws:UrlInfoResponse>

Traffic Info

http://awis.amazonaws.com?
AWSAccessKeyId=zzzzzzz
&Action=TrafficHistory
&ResponseGroup=History
&Timestamp=2007-08-07T21%3A03%3A57.000Z
&Signature=yY7tsv7tn1kk4JaJQhYkT4hd6JE%3D
&Range=5
&Url=http://www.harvard.edu/

<?xml version="1.0"?>
<aws:TrafficHistoryResponse xmlns:aws="http://alexa.amazonaws.com/doc/2005-10-05/">
    <aws:Response xmlns:aws="http://awis.amazonaws.com/doc/2005-07-11">
        <aws:OperationRequest>
            <aws:RequestId>49ab4788-19d4-8103-7dee-8ea7e7365f60</aws:RequestId>
        </aws:OperationRequest>
        <aws:TrafficHistoryResult>
            <aws:Alexa>
                <aws:TrafficHistory>
                    <aws:Range>5</aws:Range>
                    <aws:Site>http%3A%2F%2Fwww.harvard.edu%2F</aws:Site>
                    <aws:Start>2009-11-11</aws:Start>
                    <aws:HistoricalData>
                        <aws:Data>
                            <aws:Date>2009-11-11</aws:Date>
                            <aws:PageViews>
                                <aws:PerMillion>31.8</aws:PerMillion>
                                <aws:PerUser>3.5</aws:PerUser>
                            </aws:PageViews>
                            <aws:Rank>1393</aws:Rank>
                            <aws:Reach>
                                <aws:PerMillion>920</aws:PerMillion>
                            </aws:Reach>
                        </aws:Data>
                        <aws:Data>
                            <aws:Date>2009-11-12</aws:Date>
                            <aws:PageViews>
                                <aws:PerMillion>30.400000000000002</aws:PerMillion>
                                <aws:PerUser>3.4</aws:PerUser>
                            </aws:PageViews>
                            <aws:Rank>1460</aws:Rank>
                            <aws:Reach>
                                <aws:PerMillion>890</aws:PerMillion>
                            </aws:Reach>
                        </aws:Data>
                        <aws:Data>
                            <aws:Date>2009-11-13</aws:Date>
                            <aws:PageViews>
                                <aws:PerMillion>24.4</aws:PerMillion>
                                <aws:PerUser>3.1</aws:PerUser>
                            </aws:PageViews>
                            <aws:Rank>1792</aws:Rank>
                            <aws:Reach>
                                <aws:PerMillion>770</aws:PerMillion>
                            </aws:Reach>
                        </aws:Data>
                        <aws:Data>
                            <aws:Date>2009-11-14</aws:Date>
                            <aws:PageViews>
                                <aws:PerMillion>20.799999999999997</aws:PerMillion>
                                <aws:PerUser>3.0</aws:PerUser>
                            </aws:PageViews>
                            <aws:Rank>2055</aws:Rank>
                            <aws:Reach>
                                <aws:PerMillion>670</aws:PerMillion>
                            </aws:Reach>
                        </aws:Data>
                        <aws:Data>
                            <aws:Date>2009-11-15</aws:Date>
                            <aws:PageViews>
                                <aws:PerMillion>21.5</aws:PerMillion>
                                <aws:PerUser>3.2</aws:PerUser>
                            </aws:PageViews>
                            <aws:Rank>2070</aws:Rank>
                            <aws:Reach>
                                <aws:PerMillion>660</aws:PerMillion>
                            </aws:Reach>
                        </aws:Data>
                    </aws:HistoricalData>
                </aws:TrafficHistory>
            </aws:Alexa>
        </aws:TrafficHistoryResult>
        <aws:ResponseStatus xmlns:aws="http://alexa.amazonaws.com/doc/2005-10-05/">
            <aws:StatusCode>Success</aws:StatusCode>
        </aws:ResponseStatus>
    </aws:Response>
</aws:TrafficHistoryResponse>

Yahoo! Traffic Web Services (and Justin Bieber!?)

bieber

Traffic Web Services from Yahoo!

Parameters:

<?xml version="1.0"?>
<ResultSet xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="urn:yahoo:maps"
	xsi:schemaLocation="urn:yahoo:maps http://local.yahooapis.com/MapsService/V1/TrafficDataResponse.xsd">
	<LastUpdateDate>1289950985</LastUpdateDate>
	<Result type="incident">
		<Title>Slow traffic, on MASS PIKE WB at MARKET ST OVPS</Title>
		<Description>SLOW TRAFFIC; DELAYS; ALLOW EXTRA TIME.</Description>
		<Latitude>42.358129</Latitude>
		<Longitude>-71.147829</Longitude>
		<Direction>WB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289948179</ReportDate>
		<UpdateDate>1289950482</UpdateDate>
		<EndDate>1289952283</EndDate>
	</Result>
	<Result type="incident">
		<Title>Slow traffic, on I-93 SB at BUNKER HILL COMMUNITY COLLEGE</Title>
		<Description>SLOW TRAFFIC; MODERATE CONGESTION; ALLOW EXTRA TIME.</Description>
		<Latitude>42.374160</Latitude>
		<Longitude>-71.070980</Longitude>
		<Direction>SB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289949401</ReportDate>
		<UpdateDate>1289949776</UpdateDate>
		<EndDate>1289951577</EndDate>
	</Result>
	<Result type="incident">
		<Title>Slow traffic, on I-93 SB at LEVERETT CONNECTOR</Title>
		<Description>SLOW TRAFFIC; HEAVY CONGESTION; ALLOW EXTRA TIME.</Description>
		<Latitude>42.367987</Latitude>
		<Longitude>-71.062543</Longitude>
		<Direction>SB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289935634</ReportDate>
		<UpdateDate>1289950482</UpdateDate>
		<EndDate>1289952283</EndDate>
	</Result>
	<Result type="incident">
		<Title>Slow traffic, on O NEILL TUNL SB at ZAKIM BUNKER HILL BRG</Title>
		<Description>SLOW TRAFFIC; HEAVY CONGESTION; ALLOW EXTRA TIME.</Description>
		<Latitude>42.368100</Latitude>
		<Longitude>-71.062300</Longitude>
		<Direction>SB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289932363</ReportDate>
		<UpdateDate>1289949422</UpdateDate>
		<EndDate>1289951223</EndDate>
	</Result>
	<Result type="incident">
		<Title>Slow traffic, on O NEILL TUNL NB at TD GARDEN</Title>
		<Description>CONCERT; JUSTIN BIEBER; EXPECT DELAYS.</Description>
		<Latitude>42.366744</Latitude>
		<Longitude>-71.060759</Longitude>
		<Direction>NB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289692800</ReportDate>
		<UpdateDate>1289869277</UpdateDate>
		<EndDate>1289966400</EndDate>
	</Result>
	<Result type="incident">
		<Title>Slow traffic, on SOUTHEAST EXPY SB at O NEILL TUNNEL EXIT</Title>
		<Description>SLOW TRAFFIC; HEAVY CONGESTION; ALLOW EXTRA TIME.</Description>
		<Latitude>42.350060</Latitude>
		<Longitude>-71.059520</Longitude>
		<Direction>SB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289950374</ReportDate>
		<UpdateDate>1289950829</UpdateDate>
		<EndDate>1289952630</EndDate>
	</Result>
	<Result type="incident">
		<Title>Slow traffic, on RT-1 NB at SARGENT ST</Title>
		<Description>SLOW TRAFFIC; DELAYS; ALLOW EXTRA TIME.</Description>
		<Latitude>42.421810</Latitude>
		<Longitude>-71.027540</Longitude>
		<Direction>NB</Direction>
		<Severity>2</Severity>
		<ReportDate>1289947250</ReportDate>
		<UpdateDate>1289949422</UpdateDate>
		<EndDate>1289951223</EndDate>
	</Result>
</ResultSet>
        

Yahoo! Search

Yahoo! Web Search

Parameters:

<ResultSet xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:yahoo:srch" xsi:schemaLocation="urn:yahoo:srch http://api.search.yahoo.com/WebSearchService/V1/WebSearchResponse.xsd" type="web" totalResultsAvailable="62600000" totalResultsReturned="5" firstResultPosition="1" moreSearch="/WebSearchService/V1/webSearch?query=semantic+web&amp;appid=vFOFNRHV34FYD71puwTnzWb7hOTcfc8PmJxeeMiOTWjqmMdGjxCw2jsqymHXZdIX5Ro-&amp;region=us">
    <Result>
        <Title>Semantic Web - Wikipedia, the free encyclopedia</Title>
        <Summary>Semantic Web is a group of methods and technologies to allow machines to understand the meaning - or "semantics" - of information on the World Wide Web.[1] ...</Summary>
        <Url>http://en.wikipedia.org/wiki/Semantic_Web</Url>
        <ClickUrl>http://en.wikipedia.org/wiki/Semantic_Web</ClickUrl>
        <DisplayUrl>en.wikipedia.org/wiki/Semantic_Web</DisplayUrl>
        <ModificationDate>1289026800</ModificationDate>
        <MimeType>text/html</MimeType>
        <Cache>
            <Url>http://uk.wrs.yahoo.com/_ylt=A0WTecHwtuJM3CsBwC_dmMwF;_ylu=X3oDMTBwOHA5a2tvBGNvbG8DdwRwb3MDMQRzZWMDc3IEdnRpZAM-/SIG=1877s2k04/EXP=1290012784/**http%3A//66.218.69.11/search/cache%3Fei=UTF-8%26appid=vFOFNRHV34FYD71puwTnzWb7hOTcfc8PmJxeeMiOTWjqmMdGjxCw2jsqymHXZdIX5Ro-%26query=semantic%2Bweb%26results=5%26u=en.wikipedia.org/wiki/Semantic_Web%26w=semantic%26d=JRXcnVtsVxci%26icp=1%26.intl=us</Url>
            <Size>121217</Size>
        </Cache>
    </Result>

    <Result>
        <Title>W3C Semantic Web Activity</Title>
        <Summary>A list of events, publications, and presentations, and discussions.</Summary>
        <Url>http://www.w3.org/2001/sw/</Url>
        <ClickUrl>http://www.w3.org/2001/sw/</ClickUrl>
        <DisplayUrl>www.w3.org/2001/sw/</DisplayUrl>
        <ModificationDate>1277881200</ModificationDate>
        <MimeType>text/html</MimeType>

        <Cache>
            <Url>http://uk.wrs.yahoo.com/_ylt=A0WTecHwtuJM3CsBwy_dmMwF;_ylu=X3oDMTBwZG5hOWwzBGNvbG8DdwRwb3MDMgRzZWMDc3IEdnRpZAM-/SIG=17oio8lin/EXP=1290012784/**http%3A//66.218.69.11/search/cache%3Fei=UTF-8%26appid=vFOFNRHV34FYD71puwTnzWb7hOTcfc8PmJxeeMiOTWjqmMdGjxCw2jsqymHXZdIX5Ro-%26query=semantic%2Bweb%26results=5%26u=www.w3.org/2001/sw/%26w=semantic%26d=Bm_2fFtsVwZr%26icp=1%26.intl=us</Url>
            <Size>27640</Size>
        </Cache>
    </Result>

    <Result>
        <Title>semanticweb.com - The Voice of Semantic Web Business</Title>
        <Summary>The Voice of Semantic Web Business ... Good news on the money front: Two companies in the semantic web and big data sets space have come into some additional funding. ...</Summary>
        <Url>http://semanticweb.com/</Url>
        <ClickUrl>http://semanticweb.com/</ClickUrl>
        <DisplayUrl>semanticweb.com/</DisplayUrl>
        <ModificationDate>1289462400</ModificationDate>
        <MimeType>text/html</MimeType>
    </Result>

    <Result>
        <Title>semanticweb.org</Title>
        <Summary>Collaborative research projects to demonstrate ideas and concepts leading to the Semantic Web, a vision of the next generation Internet as intelligently linked, agent-driven, structured collections of machine-readable information.</Summary>
        <Url>http://www.semanticweb.org/</Url>
        <ClickUrl>http://www.semanticweb.org/</ClickUrl>
        <DisplayUrl>www.semanticweb.org/</DisplayUrl>
        <ModificationDate>1289548800</ModificationDate>
        <MimeType>text/html</MimeType>

        <Cache>
            <Url>http://uk.wrs.yahoo.com/_ylt=A0WTecHwtuJM3CsByC_dmMwF;_ylu=X3oDMTBwYzFnNmkxBGNvbG8DdwRwb3MDNARzZWMDc3IEdnRpZAM-/SIG=17p943tr0/EXP=1290012784/**http%3A//66.218.69.11/search/cache%3Fei=UTF-8%26appid=vFOFNRHV34FYD71puwTnzWb7hOTcfc8PmJxeeMiOTWjqmMdGjxCw2jsqymHXZdIX5Ro-%26query=semantic%2Bweb%26results=5%26u=www.semanticweb.org/%26w=semantic%26d=FNO3iVtsVzee%26icp=1%26.intl=us</Url>
            <Size>22627</Size>
        </Cache>
    </Result>

    <Result>
        <Title>Semantic Web: Information from Answers.com</Title>
        <Summary>Semantic Web A collaboration of the World Wide Web Consortium (W3C) and others to provide a standard for defining data on the Web</Summary>
        <Url>http://www.answers.com/topic/semantic-web</Url>
        <ClickUrl>http://www.answers.com/topic/semantic-web</ClickUrl>
        <DisplayUrl>www.answers.com/topic/semantic-web</DisplayUrl>
        <ModificationDate>1289548800</ModificationDate>
        <MimeType>text/html</MimeType>
        <Cache>
            <Url>http://uk.wrs.yahoo.com/_ylt=A0WTecHwtuJM3CsByy_dmMwF;_ylu=X3oDMTBwdDJ0dDBoBGNvbG8DdwRwb3MDNQRzZWMDc3IEdnRpZAM-/SIG=187h1oa7s/EXP=1290012784/**http%3A//66.218.69.11/search/cache%3Fei=UTF-8%26appid=vFOFNRHV34FYD71puwTnzWb7hOTcfc8PmJxeeMiOTWjqmMdGjxCw2jsqymHXZdIX5Ro-%26query=semantic%2Bweb%26results=5%26u=www.answers.com/topic/semantic-web%26w=semantic%26d=LoBnXltsVzPM%26icp=1%26.intl=us</Url>
            <Size>159664</Size>
        </Cache>
    </Result>

</ResultSet>

Platforms Driven by Web Services

Portals and Web Services at Harvard

portal

Choices, choices, choices

The National Weather Service (National Digital Forecast Database XML/SOAP Service - NOAA) makes data available in:

Web Services and Legacy Systems

ws

Harvard University Course Catalog

course catalog

course catalog

Harvard University Course Catalog

course catalog architecture

Data from Schools

<courses xmlns="http://icommons.harvard.edu/Schema">
<course academic_year="2010" course_code="CSCI_E-12" school_id="dce" section="(21144)" term_code="2">
    <catalog_info>
      <title>Fundamentals of Website Development</title>
      <prereq>see course description</prereq>
      <enrollment_limit_flag>false</enrollment_limit_flag>
      <description>This course provides a comprehensive overview of website development. Students explore the prevailing vocabulary, tools, and standards used in the field and learn how the various facets5;including XHTML, CSS, JavaScript, Ajax, multimedia, scripting languages, HTTP, clients, servers, and databases5;function together in today's web environment. The course provides a solid web development foundation, focusing on content and client-side (browser) components (XHTML, CSS, JavaScript, multimedia), with an overview of the server-side technologies. In addition, software and services that are easily incorporated into a website (for example, maps, checkout, blogs, content management) are surveyed and discussed. Students produce an interactive website on the topic of their choice for the final project and leave the course prepared for more advanced and focused web development studies. Prerequisites: CSCI E-1, or the equivalent experience.
      </description>
      <faculty_description>Mr David P. Heitmeyer</faculty_description>
      <department_short_name>Computer Science</department_short_name>
      <credits>4</credits>
      <crosslisted_flag>false</crosslisted_flag>
      <crossregistration_flag>false</crossregistration_flag>
      <crossregistration_description>Not eligible for cross-registration.</crossregistration_description>
      <undergraduate_credit_flag>false</undergraduate_credit_flag>
      <graduate_credit_flag>true</graduate_credit_flag>
      <audit_flag>N</audit_flag>
      <offered_flag>true</offered_flag>
      <exclude_from_catalog>false</exclude_from_catalog>
      <meeting_schedule>
        <meeting days_of_week="Wednesday" end_time="19:30:00" location="Maxwell Dworkin G115" start_time="17:30:00" />
      </meeting_schedule>
      <url>http://www.extension.harvard.edu/2010-11/courses/syllabi/21144</url>
    </catalog_info>
    <staff>
      <person id="xxxxxxxx" seniority_sort="1">
        <display_name>David P. Heitmeyer</display_name>
      </person>
    </staff>
  </course>
  </courses>

Apache Solr - Search and Facets

Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of many of the world's largest internet sites.

Facets

facet

facet

Data to Solr

<add xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:c="http://icommons.harvard.edu/Schema"
	xmlns:cracc="http://cracc.isites.harvard.edu">
	<doc>
		<field name="id">d_dce_1011_1_CSCI_E-153_12151</field>
		<field name="course_time">7:35 p.m. - 9:35 p.m.</field>
		<field name="course_time_sort">19.583333333333333333</field>
		<field name="day_period">Evening (after 5pm)</field>
		<field name="course_days">Tu</field>

		<field name="course_meeting_days">Tuesday</field>
		<field name="course_days_sort">BZ</field>
		<field name="day">Tuesday</field>
		<field name="school_short_name">dce</field>
		<field name="school_nm">Harvard Extension School</field>
		<field name="dept_area_category">Computer Science</field>

		<field name="course_no">CSCI E-153</field>
		<field name="course_title">Web Development Using XML</field>
		<field name="course_desc">Students learn key XML technologies (XML, XPath,XSL, XML Schema, RNG, DTD, XQuery, DOM) as well as specific markup languages relevant to website development (XHTML, XHTMI Mobile Profile, RSS, RDF, XSL-FO, SVG, DocBook, OOXML, OpenDocument, XForms). In addition, the course covers topics such as XML and databases (native XML databases and RDBMS), XML programming APIs (DOM and SAX), Apache Cocoon (an open-source XML publishing framework), and the role of XML in Web 2.0 to deliver data and functionality through Ajax and web services (SOAP and REST).  Using these technologies, students develop dynamic, data-driven websites that are capable of delivering content in a variety of media formats (screen, text, print, graphics) to a variety of devices (desktop, handheld, mobile) for a variety of audiences. Prerequisite: CSCI E-12, or the equivalent experience.</field>
		<field name="fas_cat_no" />
		<field name="credits">4</field>
		<field name="xreg_fl">Not eligible for cross-registration</field>

		<field name="xlist_fl">false</field>
		<field name="special_notes" />
		<field name="enroll_limit_fl">N</field>
		<field name="audit_fl" />
		<field name="semester_yr">DCEFALL2010</field>
		<field name="term_desc">Fall 2010</field>
		<field name="coordinated_semester_yr">Sep to Dec 2010 (Fall Term)</field>

		<field name="offer_yr" />
		<field name="instructor">Mr David P. Heitmeyer</field>
		<field name="section_no">(12151)</field>
		<field name="exam_group" />
		<field name="prereq">see course description</field>
		<field name="start_time" />
		<field name="location">1 Story Street 306</field>

		<field name="x_sched" />
		<field name="y_sched" />
		<field name="xlist_school_short_name" />
		<field name="xlist_school_nm" />
		<field name="xlist_course_no" />
		<field name="grad_undergrad_fl" />
		<field name="syllabus_url">http://www.extension.harvard.edu/2010-11/courses/syllabi/12151</field>
	</doc>

	<doc>...</doc>
	<doc>...</doc>
	<doc>...</doc>
	<doc>...</doc>
</add>

Data from Solr

 <response>
    <lst name="responseHeader">
        <int name="status">0</int>
        <int name="QTime">156</int>
        <lst name="params">
            <str name="facet">on</str>
            <str name="sort">course_title asc</str>
            <str name="facet.mincount">1</str>
            <str name="facet.limit">-1</str>
            <str name="wt">standard</str>
            <str name="version">2.2</str>
            <str name="rows">25</str>
            <str name="fl">*,score</str>
            <str name="start">0</str>
            <str name="q">Economics</str>
            <arr name="facet.field">
                <str>coordinated_semester_yr</str>
                <str>term_desc</str>
                <str>school_nm</str>
                <str>dept_area_category</str>
                <str>day</str>
                <str>day_period</str>
                <str>xreg_fl</str>
            </arr>
            <str name="qt">standard</str>
        </lst>
    </lst>
    <result name="response" numFound="638" start="0" maxScore="0.97167623">
        <doc>
            <float name="score">0.5830058</float>
            <str name="audit_fl"/>
            <arr name="coordinated_semester_yr">
                <str>Jan to May 2011 (Spring Term)</str>
            </arr>
            <str name="course_days">TuTh</str>
            <str name="course_days_sort">BDZ</str>
            <str name="course_desc">Analyses the libertarian perspective on economic and social policy. This perspective differs from both liberal and conservative views, arguing for minimal government in most arenas. Policies addressed include drug prohibition, gun control, public education, abortion rights, gay marriage, income redistribution, and campaign finance regulation.</str>
            <str name="course_meeting_days">TuesdayThursday</str>
            <str name="course_no">ECON 1017</str>
            <str name="course_time">10:00 a.m. - 11:30 a.m.</str>
            <float name="course_time_sort">10.0</float>
            <str name="course_title">A Libertarian Perspective on Economic and Social Policy</str>
            <str name="credits">Half course</str>
            <arr name="day">
                <str>Tuesday</str>
                <str>Thursday</str>
            </arr>
            <arr name="day_period">
                <str>Morning (8am-Noon)</str>
            </arr>
            <str name="dept_area_category">Economics</str>
            <str name="enroll_limit_fl">N</str>
            <str name="exam_group"/>
            <str name="fas_cat_no"/>
            <str name="grad_undergrad_fl">B</str>
            <str name="id">d_fas_1011_2_ECON_1017_2</str>
            <str name="instructor">Jeffrey A. Miron</str>
            <str name="location">TBA</str>
            <str name="offer_yr"/>
            <str name="prereq">Social Analysis 10, or Economics 10 (fall term) and concurrent enrollment in Economics 10 (spring term).</str>
            <str name="school_nm">Faculty of Arts and Sciences</str>
            <str name="school_short_name">fas</str>
            <str name="section_no">2</str>
            <str name="semester_yr">FASSPRING2011</str>
            <str name="special_notes"/>
            <str name="start_time"/>
            <str name="syllabus_url">http://www.courses.fas.harvard.edu/1197</str>
            <str name="term_desc">Spring 2011</str>
            <str name="x_sched"/>
            <str name="xlist_course_no"/>
            <str name="xlist_fl">false</str>
            <str name="xlist_school_nm"/>
            <str name="xlist_school_short_name"/>
            <str name="xreg_fl">Eligible for cross-registration</str>
            <str name="y_sched"/>
        </doc>
        <doc>...</doc>
        <doc>...</doc>
        <doc>...</doc>
        <doc>...</doc>
    </result>
    <lst name="facet_counts">
        <lst name="facet_queries"/>
        <lst name="facet_fields">
            <lst name="coordinated_semester_yr">
                <int name="Jan 2011 (Winter Session)">9</int>
                <int name="Jan to May 2011 (Spring Term)">284</int>
                <int name="Jun to Aug 2010 (Summer Term)">53</int>
                <int name="Sep to Dec 2010 (Fall Term)">308</int>
            </lst>
            <lst name="term_desc">
                <int name="2010 Summer Semester">53</int>
                <int name="Fall 1 2010">4</int>
                <int name="Fall 2 2010">5</int>
                <int name="Fall 2010">265</int>
                <int name="Full Year 2010">10</int>
                <int name="Spring 1 2011">5</int>
                <int name="Spring 2 2011">4</int>
                <int name="Spring 2010">40</int>
                <int name="Spring 2011">201</int>
                <int name="Summer 1 2010">7</int>
                <int name="Summer 2 2010">2</int>
                <int name="Winter 2011">9</int>
                <int name="Year Long 2010- (2nd half)">33</int>
            </lst>
            <lst name="school_nm">
                <int name="Faculty of Arts and Sciences">288</int>
                <int name="Harvard Business School - Doctoral Program">12</int>
                <int name="Harvard Divinity School">9</int>
                <int name="Harvard Extension School">45</int>
                <int name="Harvard Graduate School of Design">28</int>
                <int name="Harvard Graduate School of Education">15</int>
                <int name="Harvard Kennedy School">84</int>
                <int name="Harvard Law School">62</int>
                <int name="Harvard Medical School">9</int>
                <int name="Harvard School of Public Health">33</int>
                <int name="Harvard Summer School">53</int>
            </lst>
            <lst name="dept_area_category">
                <int name="">113</int>
                <int name="Aesthetic and Interpretive Understanding">1</int>
                <int name="African and African American Studies">7</int>
                <int name="Anthropology">3</int>
                <int name="Anthropology and Archaeology">2</int>
                <int name="Applied Mathematics">3</int>
                <int name="Arabic">1</int>
                <int name="Architecture">8</int>
                <int name="Arts in Education">1</int>
                <int name="BCMP">1</int>
                <int name="Biological Sciences">2</int>
                <int name="Chinese">2</int>
                <int name="Computer Science">8</int>
                <int name="Culture and Belief">1</int>
                <int name="Culture, Communities, and Contexts">1</int>
                <int name="Development in Specific Age Periods">1</int>
                <int name="Diversity and Equity">2</int>
                <int name="Earth and Planetary Sciences">1</int>
                <int name="Economics">187</int>
                <int name="Economics of Education">5</int>
                <int name="Engineering Sciences">1</int>
                <int name="English">2</int>
                <int name="Environmental Health">3</int>
                <int name="Environmental Science and Public Policy">2</int>
                <int name="Environmental Studies">6</int>
                <int name="Epidemiology">1</int>
                <int name="French">1</int>
                <int name="Freshman Seminar">9</int>
                <int name="German">1</int>
                <int name="Global Health &amp; Population">15</int>
                <int name="Government">35</int>
                <int name="Health Policy">2</int>
                <int name="Health Policy &amp; Management">9</int>
                <int name="History">26</int>
                <int name="History and Literature">3</int>
                <int name="History of Art and Architecture">4</int>
                <int name="History of Science">1</int>
                <int name="History, Philosophy, and Foundations of Education">2</int>
                <int name="International Education">1</int>
                <int name="Journalism">1</int>
                <int name="Landscape Architecture">7</int>
                <int name="Latin American Studies">1</int>
                <int name="Law">62</int>
                <int name="Life Sciences">1</int>
                <int name="Literature">1</int>
                <int name="Management">19</int>
                <int name="Mathematics">5</int>
                <int name="Mind, Brain, and Behavior">1</int>
                <int name="Nutrition">1</int>
                <int name="OEB">2</int>
                <int name="Philosophy">1</int>
                <int name="Psychology">2</int>
                <int name="Science of Living Systems">1</int>
                <int name="Science of the Physical Universe">1</int>
                <int name="Slavic">2</int>
                <int name="Soc, Human Devlp, and Health">4</int>
                <int name="Social Analysis">2</int>
                <int name="Social Studies">9</int>
                <int name="Societies of the World">5</int>
                <int name="Sociology">8</int>
                <int name="Sociology of Education">1</int>
                <int name="Spanish">2</int>
                <int name="Statistics">4</int>
                <int name="Studies of Women, Gender, and Sexuality">2</int>
                <int name="Ukrainian">1</int>
                <int name="Undergrad Medical Education">2</int>
                <int name="United States in the World">3</int>
                <int name="Urban Planning and Design">13</int>
            </lst>
            <lst name="day">
                <int name="Friday">96</int>
                <int name="Monday">219</int>
                <int name="Saturday">3</int>
                <int name="Thursday">195</int>
                <int name="Tuesday">205</int>
                <int name="Wednesday">214</int>
            </lst>
            <lst name="day_period">
                <int name="Afternoon (Noon-5pm)">311</int>
                <int name="Evening (after 5pm)">55</int>
                <int name="Morning (8am-Noon)">178</int>
            </lst>
            <lst name="xreg_fl">
                <int name="Eligible for cross-registration">515</int>
                <int name="Not eligible for cross-registration">123</int>
            </lst>
        </lst>
        <lst name="facet_dates"/>
    </lst>

</response>
 

Course Data API for Mobile

start =
  element response {
    element courses {
      attribute numFound { xsd:integer },
      element course {
        attribute id { xsd:NCName },
        element school_name { text },
        element title { text },
        element course_number { text },
        element section_number { text },
        element location { text },
        element description { text },
        element faculty_description { text },
        element term_description { text },
        element calendar_period { text },
        element department { text },
        element meeting_time { text },
        element credits { text },
        element crossreg { text },
        element exam_group { empty },
        element notes { empty },
        element prereq { text },
        element url { xsd:anyURI }
      }+
    },
    element facets {
      element facet {
        attribute name { xsd:NCName },
        element field {
          attribute count { xsd:integer },
          attribute name { text },
          attribute short_name { text }?
        }+
      }+
    }
  }

Textbook Information

XML-RPC style POST to bookstore for textbook information:

<textbookorder>
  <courses>
    <course dept="CSCI E" num="153" sect="01" term="F10" />
  </courses>
</textbookorder>

A Web Application with Amazon Web Services

Live Site
Congress

Congress with Amazon WS

Female Members of the House Appropriations Committee:
select * from Congress where Gender = 'F' and Committee = 'House Appropriations'
webapp

Senators:
select * from Congress where Type = 'sen'
webapp

Senator Scott Brown:
select * from Congress where Id = '412384'
webapp

Amazon Web Services

Information from Amazon about their Web Services:


Amazon S3 (Simple Storage Service)

AWS SimpleDB

SimpleDB is for storing structured data (items with attribute and values) and for querying and retrieving this data. REST and SOAP interfaces are provided, and libraries for many languages are available (PHP, Java, C#, Ruby, Perl, Python).

The form of the structured data is flexible -- an item can have attributes, and these attributes can be multi-valued. Items within SimpleDB do not have to have the same set of attributes.

The sample data used in the SimpleDB documentation is:
sampledata

SimpleDB and Congress

We're going to put the Congress data into Simple DB. The photos will be stored in S3.

As an example, Scott Brown would have the following attributes and values:

AttributeValue
FullNameSen. Scott Brown [R, MA]
LastNameBrown
FirstNameScott
Typesen
StateMA
District
PartyRepublican
Birthday1959-09-12
GenderM
URLhttp://scottbrown.senate.gov/
Id
Note: maps to image
412384
OSIDN00031174
BioGuideIdB001268
StartDate2010-02-04
EndDate2012-12-31
Committee Senate Armed Services
Senate Armed Services: Airland
Senate Armed Services: Emerging Threats and Capabilities
Senate Armed Services: Strategic Forces
Senate Homeland Security and Governmental Affairs
Senate Homeland Security and Governmental Affairs: Contracting Oversight (Ad Hoc)
Senate Homeland Security and Governmental Affairs: Disaster Recovery (Ad Hoc)
Senate Homeland Security and Governmental Affairs: Oversight of Government Management, the Federal Workforce, and the District of Columbia
Senate Veterans' Affairs

SimpleDB - Add Data

HTTP Post is used to input the data to SimpleDB. The parameters submitted are shown below, as is the full HTTP transaction:

Submitted Parameters

ParameterParameter Value
AWSAccessKeyId0G0R5XVDGE4W4B763702
ActionPutAttributes
DomainNameCongress
ItemNamePerson_412384
SignatureM2z5wHTT5UgmtQ0vbseAq5dTp4fYt1b6Ak96yIGd3Xs=
SignatureMethodHmacSHA256
SignatureVersion2
Timestamp2010-11-16T18:22:11.000Z
Version2007-11-07
Attribute.1.NameFirstName
Attribute.1.ValueScott
Attribute.3.NameFullName
Attribute.3.ValueSen. Scott Brown [R, MA]
Attribute.4.NameOSID
Attribute.4.ValueN00031174
Attribute.5.NameParty
Attribute.5.ValueRepublican
Attribute.6.NameBioGuideId
Attribute.6.ValueB001268
Attribute.7.NameId
Attribute.7.Value412384
Attribute.8.NameURL
Attribute.8.Valuehttp://scottbrown.senate.gov
Attribute.9.NameGender
Attribute.9.ValueM
Attribute.10.NameBirthday
Attribute.10.Value1959-09-12
Attribute.11.NameStartDate
Attribute.11.Value2010-02-04
Attribute.12.NameEndDate
Attribute.12.Value2012-12-31
Attribute.13.NameType
Attribute.13.Valuesen
Attribute.14.NameState
Attribute.14.ValueMA
Attribute.15.NameCommittee
Attribute.15.ValueSenate Armed Services
Attribute.16.NameCommittee
Attribute.16.ValueSenate Armed Services: Airland
Attribute.17.NameCommittee
Attribute.17.ValueSenate Armed Services: Emerging Threats and Capabilities
Attribute.18.NameCommittee
Attribute.18.ValueSenate Armed Services: Strategic Forces
Attribute.19.NameCommittee
Attribute.19.ValueSenate Homeland Security and Governmental Affairs
Attribute.2.NameLastName
Attribute.2.ValueBrown
Attribute.20.NameCommittee
Attribute.20.ValueSenate Homeland Security and Governmental Affairs: Contracting Oversight (Ad Hoc)
Attribute.21.NameCommittee
Attribute.21.ValueSenate Homeland Security and Governmental Affairs: Disaster Recovery (Ad Hoc)
Attribute.22.NameCommittee
Attribute.22.ValueSenate Homeland Security and Governmental Affairs: Oversight of Government Management, the Federal Workforce, and the District of Columbia
Attribute.23.NameCommittee
Attribute.23.ValueSenate Veterans' Affairs

HTTP Transaction

POST https://sdb.amazonaws.com
Content-Type: application/x-www-form-urlencoded; charset=utf-8

Attribute.5.Name=Party&Attribute.17.Name=Committee&Attribute.1.Name=FirstName&Attribute.11.Value=2010-02-04&Action=PutAttributes&Attribute.15.Value=Senate%20Armed%20Services&SignatureMethod=HmacSHA256&Attribute.18.Name=Committee&Attribute.19.Name=Committee&Attribute.8.Name=URL&Attribute.2.Name=LastName&Attribute.20.Name=Committee&Attribute.6.Value=B001268&Attribute.23.Name=Committee&Attribute.9.Name=Gender&Version=2007-11-07&Timestamp=2010-11-16T18%3A22%3A11.000Z&Attribute.19.Value=Senate%20Homeland%20Security%20and%20Governmental%20Affairs&Attribute.10.Value=1959-09-12&Attribute.4.Value=N00031174&Attribute.22.Name=Committee&Attribute.16.Name=Committee&Attribute.9.Value=M&ItemName=Person_412384&Attribute.3.Value=Sen.%20Scott%20Brown%20%5BR%2C%20MA%5D&Attribute.7.Name=Id&Attribute.13.Value=sen&Attribute.7.Value=412384&Attribute.12.Name=EndDate&Attribute.16.Value=Senate%20Armed%20Services%3A%20Airland&Attribute.14.Value=MA&Signature=M2z5wHTT5UgmtQ0vbseAq5dTp4fYt1b6Ak96yIGd3Xs%3D&Attribute.11.Name=StartDate&Attribute.3.Name=FullName&SignatureVersion=2&Attribute.10.Name=Birthday&Attribute.1.Value=Scott&Attribute.22.Value=Senate%20Homeland%20Security%20and%20Governmental%20Affairs%3A%20Oversight%20of%20Government%20Management%2C%20the%20Federal%20Workforce%2C%20and%20the%20District%20of%20Columbia&Attribute.20.Value=Senate%20Homeland%20Security%20and%20Governmental%20Affairs%3A%20Contracting%20Oversight%20%28Ad%20Hoc%29&Attribute.14.Name=State&Attribute.13.Name=Type&Attribute.23.Value=Senate%20Veterans%27%20Affairs&Attribute.21.Value=Senate%20Homeland%20Security%20and%20Governmental%20Affairs%3A%20Disaster%20Recovery%20%28Ad%20Hoc%29&Attribute.6.Name=BioGuideId&Attribute.4.Name=OSID&Attribute.2.Value=Brown&Attribute.18.Value=Senate%20Armed%20Services%3A%20Strategic%20Forces&AWSAccessKeyId=0G0R5XVDGE4W4B763702&Attribute.17.Value=Senate%20Armed%20Services%3A%20Emerging%20Threats%20and%20Capabilities&Attribute.12.Value=2012-12-31&Attribute.21.Name=Committee&Attribute.15.Name=Committee&Attribute.5.Value=Republican&DomainName=Congress&Attribute.8.Value=http%3A%2F%2Fscottbrown.senate.gov

HTTP/1.1 200 OK
Server: Amazon SimpleDB
Content-Type: text/xml
Client-Date: Mon, 04 Nov 2010 15:17:54 GMT
Client-Peer: 72.21.206.197:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)05/CN=VeriSign Class 3 Secure Server CA
Client-SSL-Cert-Subject: /C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=sdb.amazonaws.com
Client-SSL-Cipher: RC4-MD5
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked

<?xml version="1.0"?>
<PutAttributesResponse xmlns="http://sdb.amazonaws.com/doc/2007-11-07/">
    <ResponseMetadata>
    <RequestId>357e9671-cd1f-4c3a-a989-a970afc8c6f5</RequestId>
    <BoxUsage>0.0000279489</BoxUsage></ResponseMetadata>
</PutAttributesResponse>

SimpleDB - Add Data Attributes

We forgot about trucks.
Dog's View of Washington

Let's add a "vehicle" attribute:

ParameterParameter Value
AWSAccessKeyId0G0R5XVDGE4W4B763702
ActionPutAttributes
DomainNameCongress
ItemNamePerson_412384
SignatureWU5mvSeEG9pYPONMx8Cc3Rxo17fCs/IdEdmWd69BHwU=
SignatureMethodHmacSHA256
SignatureVersion2
Timestamp2010-11-15T15:10:57.000Z
Version2007-11-07
Attribute.1.NameVehicle
Attribute.1.Value2005 GMC Canyon
POST https://sdb.amazonaws.com
Content-Type: application/x-www-form-urlencoded; charset=utf-8

Attribute.1.Name=Vehicle&ItemName=Person_412384&Attribute.1.Value=2005+GMC+Canyon&AWSAccessKeyId=0G0R5XVDGE4W4B763702&Signature=WU5mvSeEG9pYPONMx8Cc3Rxo17fCs%2FIdEdmWd69BHwU%3D&Version=2007-11-07&Timestamp=2009-01-05T15%3A10%3A57.000Z&Action=PutAttributes&DomainName=Congress&SignatureMethod=HmacSHA256&SignatureVersion=2
HTTP/1.1 200 OK
Server: Amazon SimpleDB
Content-Type: text/xml
Client-Date: Tue, 15 Nov 2010 15:10:57 GMT
Client-Peer: 207.171.168.122:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)05/CN=VeriSign Class 3 Secure Server CA
Client-SSL-Cert-Subject: /C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=sdb.amazonaws.com
Client-SSL-Cipher: RC4-MD5
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked

<?xml version="1.0"?>
<PutAttributesResponse xmlns="http://sdb.amazonaws.com/doc/2007-11-07/">
<ResponseMetadata>
<RequestId>3e94f8e8-c58b-449c-aeb9-2864ff0bda2f</RequestId>
<BoxUsage>0.0000219923</BoxUsage>
</ResponseMetadata>
</PutAttributesResponse>

SimpleDB - Get

Let's send a query (via HTTP) to SimpleDB, and we will get back XML.

ParameterParameter Value
AWSAccessKeyId0G0R5XVDGE4W4B763702
ActionSelect
DomainNameCongress
QueryExpressionselect * from Congress where LastName = 'Brown' and State = 'MA'
Signatureow41VaplnbEkUX5OiQFhIi+ZD0Q4fzfWyTPC+hZLHGk=
SignatureMethodHmacSHA256
SignatureVersion2
Timestamp2009-05-05T15:12:34.000Z
Version2007-11-07

When we query SimpleDB, we get back XML:

<SelectResponse xmlns="http://sdb.amazonaws.com/doc/2009-04-15/">
    <SelectResult>
        <Item>
            <Name>Person_412384</Name>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Homeland Security and Governmental Affairs: Contracting Oversight (Ad Hoc)</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Armed Services: Airland</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Homeland Security and Governmental Affairs: Disaster Recovery (Ad Hoc)</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Veterans' Affairs</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Armed Services</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Armed Services: Strategic Forces</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Armed Services: Emerging Threats and Capabilities</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Homeland Security and Governmental Affairs</Value>
            </Attribute>
            <Attribute>
                <Name>Committee</Name>
                <Value>Senate Homeland Security and Governmental Affairs: Oversight of Government Management, the Federal Workforce, and the District of Columbia</Value>
            </Attribute>
            <Attribute>
                <Name>OSID</Name>
                <Value>N00031174</Value>
            </Attribute>
            <Attribute>
                <Name>Type</Name>
                <Value>sen</Value>
            </Attribute>
            <Attribute>
                <Name>Party</Name>
                <Value>Republican</Value>
            </Attribute>
            <Attribute>
                <Name>BioGuideId</Name>
                <Value>B001268</Value>
            </Attribute>
            <Attribute>
                <Name>LastName</Name>
                <Value>Brown</Value>
            </Attribute>
            <Attribute>
                <Name>Birthday</Name>
                <Value>1959-09-12</Value>
            </Attribute>
            <Attribute>
                <Name>EndDate</Name>
                <Value>2012-12-31</Value>
            </Attribute>
            <Attribute>
                <Name>State</Name>
                <Value>MA</Value>
            </Attribute>
            <Attribute>
                <Name>StartDate</Name>
                <Value>2010-02-04</Value>
            </Attribute>
            <Attribute>
                <Name>FullName</Name>
                <Value>Sen. Scott Brown [R, MA]</Value>
            </Attribute>
            <Attribute>
                <Name>Gender</Name>
                <Value>M</Value>
            </Attribute>
            <Attribute>
                <Name>FirstName</Name>
                <Value>Scott</Value>
            </Attribute>
            <Attribute>
                <Name>Id</Name>
                <Value>412384</Value>
            </Attribute>
            <Attribute>
                <Name>URL</Name>
                <Value>http://scottbrown.senate.gov</Value>
            </Attribute>
            <Attribute>
                <Name>Vehicle</Name>
                <Value>2005 GMC Canyon</Value>
            </Attribute>
        </Item>
    </SelectResult>
    <ResponseMetadata>
        <RequestId>52fbdf5b-633f-ff38-1ac2-2abdb0f33cd4</RequestId>
        <BoxUsage>0.0000229792</BoxUsage>
    </ResponseMetadata>
</SelectResponse>

Querying SimpleDB

image

ToDoFinder.com

todofinder

ToDoFinder - Search Location

http://maps.google.com/maps/geo?q=Seattle&output=xml&key=XXXX

<?xml version="1.0" encoding="UTF-8" ?>
<kml xmlns="http://earth.google.com/kml/2.0">
    <Response>
        <name>Seattle</name>
        <Status>
            <code>200</code>
            <request>geocode</request>
        </Status>
        <Placemark id="p1">
            <address>Seattle, WA, USA</address>
            <AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
                <Country>
                    <CountryNameCode>US</CountryNameCode>
                    <CountryName>USA</CountryName>
                    <AdministrativeArea>
                        <AdministrativeAreaName>WA</AdministrativeAreaName>
                        <Locality>
                            <LocalityName>Seattle</LocalityName>
                        </Locality>
                    </AdministrativeArea>
                </Country>
            </AddressDetails>
            <Point>
                <coordinates>-122.3472760,47.6209730,0</coordinates>
            </Point>
        </Placemark>
    </Response>
</kml>

ToDoFinder - Search Location

Search Location Found

http://maps.google.com/maps/geo?q=1410%2030th%20Ave%20Seattle,%20WA%2098122&output=xml&key=XXXX

<?xml version="1.0" encoding="UTF-8" ?>
<kml xmlns="http://earth.google.com/kml/2.0">
    <Response>
        <name>1410 30th Ave Seattle, WA 98122</name>
        <Status>
            <code>200</code>
            <request>geocode</request>
        </Status>
        <Placemark id="p1">
            <address>1410 30th Ave, Seattle, WA 98122, USA</address>
            <AddressDetails Accuracy="8" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
                <Country>
                    <CountryNameCode>US</CountryNameCode>
                    <CountryName>USA</CountryName>
                    <AdministrativeArea>
                        <AdministrativeAreaName>WA</AdministrativeAreaName>
                        <Locality>
                            <LocalityName>Seattle</LocalityName>
                            <Thoroughfare>
                                <ThoroughfareName>1410 30th Ave</ThoroughfareName>
                            </Thoroughfare>
                            <PostalCode>
                                <PostalCodeNumber>98122</PostalCodeNumber>
                            </PostalCode>
                        </Locality>
                    </AdministrativeArea>
                </Country>
            </AddressDetails>
            <Point>
                <coordinates>-122.2933710,47.6132900,0</coordinates>
            </Point>
        </Placemark>
    </Response>
</kml>

ToDoFinder - Did you mean?

Did You Mean

http://maps.google.com/maps/geo?q=Springfield&output=xml&key=XXXX

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.0">
	<Response>
		<name>Springfield</name>
		<Status>
			<code>200</code>
			<request>geocode</request>
		</Status>
		<Placemark id="p1">
			<address>Springfield, MO, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>MO</AdministrativeAreaName>
						<Locality>
							<LocalityName>Springfield</LocalityName>
						</Locality>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-93.2870610,37.1968280,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p2">
			<address>Springfield, Florida, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>FL</AdministrativeAreaName>
						<AddressLine>Springfield</AddressLine>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-85.6078260,30.1676410,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p3">
			<address>Springfield, IL, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>IL</AdministrativeAreaName>
						<Locality>
							<LocalityName>Springfield</LocalityName>
						</Locality>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-89.6532710,39.8018070,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p4">
			<address>Springfield, Missouri, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>MO</AdministrativeAreaName>
						<AddressLine>Springfield</AddressLine>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-93.2784825,37.1877973,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p5">
			<address>Springfield, MA, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>MA</AdministrativeAreaName>
						<Locality>
							<LocalityName>Springfield</LocalityName>
						</Locality>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-72.5851360,42.1026090,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p6">
			<address>Springfield, Virginia, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>VA</AdministrativeAreaName>
						<AddressLine>Springfield</AddressLine>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-77.3554130,38.7771426,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p7">
			<address>Springfield, OH, USA</address>
			<AddressDetails Accuracy="4" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>OH</AdministrativeAreaName>
						<Locality>
							<LocalityName>Springfield</LocalityName>
						</Locality>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-83.8067110,39.9243890,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p8">
			<address>Springfield Ave, Paris, IL 61944, USA</address>
			<AddressDetails Accuracy="6" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>IL</AdministrativeAreaName>
						<Locality>
							<LocalityName>Paris</LocalityName>
							<Thoroughfare>
								<ThoroughfareName>Springfield Ave</ThoroughfareName>
							</Thoroughfare>
							<PostalCode>
								<PostalCodeNumber>61944</PostalCodeNumber>
							</PostalCode>
						</Locality>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-87.8629470,39.6438310,0</coordinates>
			</Point>
		</Placemark>
		<Placemark id="p9">
			<address>Springfield, Baldwin, MI 49304, USA</address>
			<AddressDetails Accuracy="6" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">
				<Country>
					<CountryNameCode>US</CountryNameCode>
					<CountryName>USA</CountryName>
					<AdministrativeArea>
						<AdministrativeAreaName>MI</AdministrativeAreaName>
						<Locality>
							<LocalityName>Baldwin</LocalityName>
							<Thoroughfare>
								<ThoroughfareName>Springfield</ThoroughfareName>
							</Thoroughfare>
							<PostalCode>
								<PostalCodeNumber>49304</PostalCodeNumber>
							</PostalCode>
						</Locality>
					</AdministrativeArea>
				</Country>
			</AddressDetails>
			<Point>
				<coordinates>-85.7925660,43.8798160,0</coordinates>
			</Point>
		</Placemark>
	</Response>
</kml>

ToDoFinder - Location + Events

Copyright © 2002-2010 David P. Heitmeyer

Bookmark and Share