Google REST Web Service Security C# URL Authentication

Google URL Signing

 

To access certain Google API web services a Client ID and a digital signature is required.  A Client ID and a cryptographic key (used to generate a unique digital signature) are provided by Google once consumers have created an account.  Before a REST web service request can be made, a digital signature must be generated and passed in as part of the URL, this process is known as URL signing.  The cryptographic key, sometimes called shared secret/signing key must be kept secret and is never passed in as part of a URL request.

The following are required before a digital signature can be generated:

• A signing string (made up of the path and query of the request URL).
• A signing key (cryptographic key provided by Google).
• The correct encryption algorithm (Hash Message Access Control eg. HMAC SHA1, HMAC SHA2).

To generate a digital signature requires the following 3 steps.

1)  Construct request URL, the URL must be percent-encoded/URL encoded to ensure it is valid.

Constructing a valid URL

A URL is made up of reserved and unreserved characters.  Reserved characters in a URL have a specific purpose, for example a ‘/ ’ separates the different parts of a URL.  If a ‘/ ’ is required in a URL for any other purpose, then it must be percent-encoded to distinguish it from its reserved counterparts.  Percent-encoded reserved characters are always preceded by a ‘%’ to indicate percent-encoding.

The following are reserved characters:

!  * ' ( ) ; : @ & = + $ , / ? % # [ ]

To percent-encode a URL’s reserved characters, use the pair of hex digits which corresponds to each reserved character’s ASCII value, finish by preceding each hex digit pair with a ‘%’.  If any reserved character is non-ASCII then use the pair of hex digits which corresponds to its UTF-8 character instead.

For example, to percent-encode a '/ ':

a)  convert to or look up the ASCII value for a ‘/ ’, the ASCII value is 47

b)  next look up the pair of hex digits which correspond to this ASCII value, the hex digit pair is 2F

c)  next precede with a ‘%’, the result is %2F

URL encoding the following URL:

http://maps.googleapis.com/maps/api/geocode/json ?address=East 25th St & 3rd Ave&sensor=false&client=yourClientID

The address parameter contains spaces, spaces are not allowed in a URL, it also contains a '&'.  This URL must be encoded by replacing spaces with a '+', and replacing '&' with the two hex digits corresponding to its ASCII value.

Encoding would produce the following URL:

http://maps.googleapis.com/maps/api/geocode/json?address=East+25th+St+%26+3rd+Ave&sensor=false&client=yourClientID

For more information on ASCII characters and their corresponding pair of hex digits see the following link:

http://en.wikipedia.org/wiki/American_Standard_Code_for_Information_Interchange

To see a list of reserved/unreserved characters and additional information on URL usage, go to the following link.

http://tools.ietf.org/html/rfc3986#section-2.2

2)  Using the encoded URL

http://maps.googleapis.com/maps/api/geocode/json?address=East+25th+St+%26+3rd+Ave&sensor=false&client=yourClientID

Omit the protocol and host, leaving only the path and query, now you have the signing string.

/maps/api/geocode/json?address=East+25th+St+%26+3rd+Ave&sensor=false&client=yourClientID

3)  Generate signature using signing key, signing string and encryption algorithm.

Google provides a C# method for developers called ‘Sign’, this method accepts two string parameters and returns the signed URL request in full as a string.  When calling this method the full URL and the signing key are passed in as the two parameters of type string, see the method below.

Google’s C# 'Sign' method code snippet

public static string Sign(string url, string keyString)

{

      ASCIIEncoding encoding = new ASCIIEncoding();

      // converting key to bytes will throw an exception, need to

      // replace '-' and  '_' characters first.

      string usablePrivateKey = keyString.Replace("-", "+").Replace("_", "/");

      byte[] privateKeyBytes = Convert.FromBase64String(usablePrivateKey);

      Uri uri = new Uri(url);

      byte[] encodedPathAndQueryBytes = encoding.GetBytes(uri.LocalPath + uri.Query);

      // compute the hash

      HMACSHA1 algorithm = new HMACSHA1(privateKeyBytes);

      byte[] hash = algorithm.ComputeHash(encodedPathAndQueryBytes);

      // convert the bytes to string and make url-safe by replacing '+' and '/' characters

      string signature = Convert.ToBase64String(hash).Replace("+", "-").Replace("/", "_");

      // Add the signature to the existing URI.

      return uri.Scheme+"://"+uri.Host+uri.LocalPath + uri.Query +"&signature=" + signature;

}

A signature parameter is attached to the end of the URL, and the fully signed request URL is returned as a string.

http://maps.googleapis.com/maps/api/geocode/json?address=East+25th+St+%26+3rd+Ave&sensor=false&client=yourClientID&signature=HMACgeneratedSignature

Using this process a request can be made to any Google web service which requires a digital signature.

To download Google's C# console application for URL signing, which includes the 'Sign' method shown above, see the following link:

http://gmaps-samples.googlecode.com/svn/trunk/urlsigning/UrlSigner.cs

In some cases SSL (Secure Socket Layer) is used to provide transport level security when making requests. In these situations change the protocol from ‘http’ to ‘https’, before making the request.

Google Maps API Web Services require URL Authentication, documentation is provided on the URL signing process, how to produce valid URL's, SSL Access and processing XML results.  See the following link for more information:

http://code.google.com/apis/maps/documentation/webservices/index.html

Yahoo REST Web Service Security

Yahoo REST Web Service Authentication

All Yahoo API web services are free to consume at present.  Once service consumers have signed up for an account with Yahoo, they are assigned an API Key/Application ID.  When making a REST web service call the API Key must be included in the URL request.

Some Yahoo API web services use basic http authentication if a username and password is required, the Delicious API from Yahoo is an example.  The http Authorization header is assigned the value of the base64 encoded username and password.  A client with the following credentials:

username = MyUserName

password = MyPsswd

Places the username and password together to create the following string, which is then encoded using base64.

MyUserName:MyPsswd

The resulting base64 encoded string is passed as a http header during the request, for example:

Authorization: Basic TXlVc2VyTmFtZTpNeVBzc3dk

The Delicious API also uses Secure Socket Layer (SSL) for transport level security, the https protocol is used instead of http when making a request.  Some service providers use HTTP-Auth and SSL in combination.

Amazon Web Services (AWS) URL Authentication

There are two types of web service requests that can be made to Amazon Web Services (AWS).

• Anonymous requests are made to freely available services using an Access Key ID which is issued when consumers sign up for an account.  This key is included as a request parameter in all requests made to AWS.

• Authenticated requests must contain a request signature, in other words the URL must be signed using a generated digital signature.

AWS Access Identifiers are an Access Key ID which is your identity/username and a Secret Access Key which is your password.  The Secret Access Key must be kept secret and is never used in requests, it is instead used together with an encryption algorithm during the signing process.  The Secret Access Key is a 40 character alphanumeric sequence issued to consumers by AWS.


A Recipe for Signing URL Request’s to AWS

1)  Construct your request URL.

The following is a dummy URL

http://aws.amazon.com/onca/xml?Service=AWSDummyService&AWSAccessKeyId=00000000&Operation=DummySearch&ResponseGroup=DummyInfo,TopSellers,NewReleases&Version=2011-05-05

2)  TimeStamp your URL.

Attach a time stamp to the end of your URL.

http://aws.amazon.com/onca/xml?Service=AWSDummyService&AWSAccessKeyId=00000000&Operation=DummySearch&ResponseGroup=DummyInfo,TopSellers,NewReleases&Version=2011-05-05&TimeStamp=2011-5-03T14:22:58Z

For more info on UTC format TimeStamp, see the link below.

http://en.wikipedia.org/wiki/Unix_time

3)  URL encode any reserved characters used by query parameters.

The ResponseGroup parameter contains comma’s and the TimeStamp parameter contains colons, comma’s and colons are reserved characters. To URL encode a reserved character, use the hex digit pair corresponding to the reserved character’s ASCII value, then precede the hex digit pair with a ‘%’.

Reserved Character              Hex Digit Pair
          ,                                        %2C
          :                                        %3A

The URL encoded dummy URL:

http://aws.amazon.com/onca/xml?Service=AWSDummyService&AWSAccessKeyId=00000000&Operation=DummySearch&ResponseGroup=DummyInfo%2CTopSellers%2CNewReleases&Version=2011-05-05&TimeStamp=2011-5-03T14%3A22%3A58Z

4)  Re-arrange URL query parameters in byte order to produce the canonical string.

Capital letters come before lower case letters. A parameter name beginning with the capital letter 'D' precedes a parameter name beginnig with 'a', the names are NOT in alphabetical order.

Dummy URL query parameters in byte order:

AWSAccessKeyId=00000000&Operation=DummySearch&ResponseGroup=DummyInfo%2CTopSellers%2CnewReleases&Service=AWSDummyService&TimeStamp=2011-5-03T14%3A22%3A58Z&Version=2011-05-05

The above is the canonicalized query string used to generate a signature.

5)  Prepend Verb, Host and Path lines (preceding canonicalized query string) with a line break.

HTTP Verb – GET, PUT, POST, DELETE or other
Host - aws.amazon.com
Path - /onca/xml (if the path is empty, use a / in its place)

Each of the verb, host and path values must be on separate lines, and they must precede the canonicalized query string.

6)  The signing string in full.

PUT
aws.amazon.com
/onca/xml
AWSAccessKeyId=00000000&Operation=DummySearch&ResponseGroup=DummyInfo%2CTopSellers%2CnewReleases&Service=AWSDummyService&TimeStamp=2011-5-03T14%3A22%3A58Z&Version=2011-05-05

7)  Compute digital signature using signing string, Secret Access/Signing Key and a hash algorithm.

The encryption algorithm can be HMAC SHA1 or SHA256. Amazon provides code samples in a variety of languages available to download for their Flexible Payment Service, the samples include code for generating digital signatures, see the following link.

http://docs.amazonwebservices.com/AmazonFPS/2008-09-17/FPSBasicGuide/index.html?APPNDX_CodeSamples.html

For more information on the Secure Hash Algorithm (SHA) see the following link.

http://en.wikipedia.org/wiki/SHA_hash_functions

8)  URL encode reserved characters in the signature.

+ and = characters in the signature must be replaced by hex digit pairs.

9)  Create a signed request by attaching the signature to the end of the request URL.

http://aws.amazon.com/onca/xml? AWSAccessKeyId=00000000&Operation=DummySearch&ResponseGroup=DummyInfo%2CTopSellers%2CnewReleases&Service=AWSDummyService&TimeStamp=2011-5-03T14%3A22%3A58Z&Version=2011-05-05&Signature=URLencodedSignature

The above recipe can be used for Amazon's Product Advertising API and Amazon's Flexible Payment Service. For more information on the product advertising API see this link.

http://docs.amazonwebservices.com/AWSECommerceService/latest/DG/index.html?CHAP_Intro_AAWS.html

Sending a signature as a URL query parameter is not the only way to sign requests, a signature can be sent instead by using the Authorization header.  See below for an explanation on how this is achieved using Amazon's S3 storage service.

Sending signed requests to Amazon S3 storage service

If the signature is sent as a header, it must be set out as follows:

Authorization: AWS AWSAccesskeyId : generatedSignature

When using an Authorization header for authentication, the format of the signing string should be as follows:

VERB + \n

+ CONTENT-MD5 header value + \n

+ CONTENT-TYPE header value + \n

+ DATE header value + \n

+ AmzHeaders names and values in lower case and in alphabetical order + \n

+ Resource

The format of a signing string when using query string authentication instead of an Authorization header involves a minimal change, replace the DATE header value with an Expires parameter value.  The Expires parameter value indicates when you want the signature to expire.

The Amzheaders are any headers that start with x-amz-.

The string to sign must be URL encoded (UTF-8 encoded).

The content-type and content-md5 values are optional, if their values are not included in the string, newlines (\n) must still be included where the values would have been placed.

The resource value is the bucket and key values separated by a '/ '.

For more information on S3 signed requests see the following link:

http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html

Working with XML in C# .NET 3.5 Using XElement, xmlTrees and LINQ to XML

XElement
The XElement class represents an XML element. All elements in XML have names, they may or may not have attributes or content.

The namespace System.Xml.Linq is required when working with XElement.

The following link provides information on the different types of content an XElement can contain.

http://msdn.microsoft.com/en-us/library/bb943882(v=VS.90).aspx

For more information on the XElement class from the Microsoft site see the following link.

http://msdn.microsoft.com/en-us/library/system.xml.linq.xelement(v=VS.90).aspx

The following link provides details on members the XElement type exposes.

http://msdn.microsoft.com/en-us/library/system.xml.linq.xelement_members(v=VS.90).aspx

To create a single XML element with no content:

XElement objXElement = new XElement("Element1");

objXElement now contains the following XML:

<Element1 />

To add content to the element:

XElement obXjElement = new XElement("Element1", "ABCD");

objXElement now contains the following XML:

<Element1>ABCD</Element1>

To create a parent element with a descendant, add a new XElement object as the content argument:

XElement objXElement = new XElement("Parent", new XElement( "ChildElement1","ABCD");

objXElement now contains the following XML:

<Parent> 

     <ChildElement1>ABCD</ChildElement1> 

</Parent>

The XML produced matches the standard XML format.

As more XElements objects are added as content arguments an xmlTree can be created.

In addition to the XElement class, there is also a XDocument class which can be used to create XML documents. The following link provides information on this class from the Microsoft site.

http://msdn.microsoft.com/en-us/library/system.xml.linq.xdocument(v=VS.90).aspx

Working with XElement is far simpler than working with XDocument objects.
 
xmlTree
 
The following code snippet creates an xmlTree.


XElement xmlTreeExample =

   new XElement(“Parents”,

           new XElement("Parent",

                   new XElement( "ChildElement1","ABCD"),

                   new XElement(“ChildElement2”,

                           new XElement(“GrandChildElement1”, “1234”)

                    )
             )
    );

xmlTreeExample now contains the following XML:

<Parents>

    <Parent>

        <ChildElement1>ABCD</ChildElement1>

        <ChildElement2>

            <GrandChildElement1>1234</GrandChildElement1>

        </ChildElement2>

    </Parent>

</Parents>

Creating an xmlTree using XElement.Parse

The above example is not the only way to create an xmlTree, XElement’s Parse method can be used to create an xmlTree from a string containing XML.

XElement xmlTreeExample2 =

   new XElement.Parse(xmlString, LoadOptions.None);

LoadOptions set to None will not preserve white spaces or retain line information. If this is required then set LoadOptions to LoadOptions.PreserveWhitespace or LoadOptions.SetLineInfo. Please note, using LoadOptions.SetLineInfo will mean taking a performance hit.

Creating an xmlTree using XElement.Load

Using the code snippet above, the XElement object ‘xmlTreeExample2’ can be saved in an XML file.

xmlTreeExample2.Save(“TestTree.xml”);

The Load method allows an XElement to be loaded from an XML file.

XElement xmlTreeExample3 = XElement.Load(“TestTree.xml”);

The Load method can also be used to load an XElement from a TextReader, xmlReader and using a stream an XElement instance can be created. See the following link for more information on these alternative methods.

http://msdn.microsoft.com/en-us/library/bb350413.aspx

The following link provides more information on creating xmlTrees

http://msdn.microsoft.com/en-us/library/bb387068(v=VS.90).aspx
 
LINQ to XML
 
LINQ to XML is an approach to programming XML in .NET using LINQ query expressions. Query results can be used as parameters in XElement and XAttribute objects when creating xmlTrees. This functionality(known as functional construction) allows xmlTrees to be transformed from one shape to another.
 
Simple LINQ to XML queries
 
XElement xmlTreeExample4 contains the following XML:

<Parents>

     <Parent>

        <ChildElement1>ABCD</ChildElement1>

        <ChildElement2>

            <GrandChildElement1>1234</GrandChildElement1>

        </ChildElement2>

    </Parent>

    <Parent>

         <ChildElement1>EFGH</ChildElement1>

         <ChildElement2>

             <GrandChildElement1>5678</GrandChildElement1>

         </ChildElement2>

    </ Parent>

</Parents>


The following code snippet shows how to retrieve the value of the element ChildElement1 from each parent element in xmlTreeExample4.

IEnumerable elementValue =

             from parent in xmlTreeExample4.Descendants(“Parent”)

             select (string) parent.Element(“ChildElement1”);
 
The following code snippet shows how to retrieve the value of the element GrandChildElement1 from each parent element in the xmlTreeExample4.

IEnumerable elementValue =

             from parent in xmlTreeExample4.Descendants(“Parent”)

             select (int) parent.Element(“ChildElement2”).Element(“GrandChildElement1”);
 
To retrieve the values of all descendants of each Parent element the following code shows how this can be achieved.

var parentData = from parent in xmlTreeExample4.Descendants(“Parent”)

select new

{

            child = parent.Element(“ChildElement1”).Value,

            grandchild = parent.Element(“ChildElement2”).Element(“GrandChildElement1”).Value

};

A for each loop can be used to loop through each object in parentData, like so

foreach (var parent in parentData)

{
             //do some processing here
}

If each parent element in xmlTreeExample4 contained an attribute called ‘ParentName’, for example

<Parent ParentName=”FirstParent”>

To retrieve this ‘ParentName’ attribute value, the following can be done

IEnumerable attributeValue =

            from parent in xmlTreeExample4.Descendants(“Parent”)

            select (string) parent.Attribute("ParentName");
 
See the Microsoft site for more information on LINQ to XML

http://msdn.microsoft.com/en-us/library/bb387098(v=VS.90).aspx

For more information on Programming C# LINQ Query Expressions see this link

http://msdn.microsoft.com/en-us/library/bb397676(v=VS.90).aspx

Developing Rest Web Services using .NET WCF

There are a number of ways to create a rest web service in visual studio.

1)Convert an existing WCF soap web service into a WCF rest web service.

The following 15 minute video shows how easy it is to accomplish this.
http://channel9.msdn.com/shows/Endpoint/endpointtv-Screencast-Building-RESTful-Services-with-WCF/

2)Download Microsoft's WCF Rest Starter Kit from the following codeplex site:

http://aspnet.codeplex.com/releases/view/24644

A white paper on developing rest in WCF using the rest starter kit can be found here:

http://msdn.microsoft.com/en-us/library/ee391967.aspx

The starter kit contains various templates to speed up the process of creating services. The templates include an Atom Feed WCF Service, AtomPub WCF Service, HTTP Plain XML WCF Service, REST Collection WCF Service and a REST Singleton WCF Service.

3)Create a WCF Rest web service from scratch using a visual studio solution containing for example the following projects:-

ASP.NET Web Application project template - this project is used to create a rest service. It contains a service contract class with operation contracts and data contracts.

A Web Service template is also added to this project, this template is a visually designed class for creating a web service. This is where the implementation code for the operation contracts of the web service will be placed.

Windows Forms Application project template - used to host the rest web service. This project contains an app.config file and a code file with service host code.

Combining Cloud Computing, SOA and Web 2.0

Service Orientated Architecture (SOA) and Web 2.0 technologies can help organizations  implement Cloud services in their IT environment.

The following diagram shows the layers of an emerging Enterprise Computing Stack.

Web 2.0 allows for building user interfaces and collaboration.  Within an organisation this technology is referred to as Enterprise Web 2.0, it can be delivered using enterprise portals such as Microsoft SharePoint, Oracle WebCenter Suite and IBM Websphere Portal.

SOA enables the building and integrating of applications improving business agility.    SOA can be used to take advantage of Cloud services and present these resources to the users via Web 2.0 technologies such as AJAX, Mashups, RSS and enterprise social computing.  SOA greatly reduces the risks when building Hybrid cloud environments by providing a more integrated on-premise and public cloud solution.  SOA also makes it easier to move services between Cloud environments.  If an organization doesn’t have an on-premise cloud, having an SOA environment helps to identify and integrate public cloud services into the existing environment.

The three Cloud computing layers shown in the diagram are:
Infrastructure as a Service (IaaS) provides hardware and system software as a service including Storage, Compute and Networking.  These services provide access to IT infrastructure components.
Platform as a Service (PaaS) provides services for creating, deploying and managing applications
Software as a Service (SaaS) is application software delivered as a service.

These Cloud layers can be implemented individually, they do not depend on one another.  SOA principles can be applied when building the Cloud layers.  SOA architectural principles for example can be used to build a service-oriented Infrastructure as a Service (IaaS) layer.

Introduction to Microsoft Windows Azure Platform

 A good introduction to Windows Azure can be found at the following link

 http://msdn.microsoft.com/en-us/library/ff803364.aspx

The Windows Azure SDK download (Sept 2010 UPDATE) is available here

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=21910585-8693-4185-826E-E658535940AA

The SDK includes an off-line test environment called 'Developer Fabric', it is a simulation of Azure(Cloud Fabric), the subscription based production environment running in Microsoft's data centers. The SDK provides dev's with APIs, tools, docs and code samples to help develop app's on Win Azure.

Also available to download is Win Azure Tools for MS Visual Studio(ver.1.2 June 2010)

http://www.blogger.com/%20http://www.microsoft.com/downloads/en/details.aspx?FamilyID=2274a0a8-5d37-4eac-b50a-e197dc340f6f&displaylang=en#SystemRequirements

This download includes Windows Azure SDK and can be used with VS2008 & VS2010. Dev's can create, configure, build, debug, run, package and deploy scalable web app's and services.

There are some known issues/bugs in this SDK, some fixes can be found at

http://code.msdn.microsoft.com/wcfazure/Wiki/View.aspx?title=KnownIssues

A Windows Azure training kit is available to download, it comes with hands-on labs, presentations and demos designed to help you learn how to use Win Azure, SQL Azure and AppFabric. The June 2010 release includes new and updated labs for VS2010.

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=413E88F8-5966-4A83-B309-53B7B77EDF78&displayLang=en

Web Developer Portals

There are 4 portals associated with Windows Azure platform, 3 are subscription based the other is free to use. A windows Live ID is required to sign-in to these portals, you can sign-up for a Live ID on any one of the portal sign-in pages.

1)Windows Azure Web Dev Portal - Subscription required, app's and services are uploaded to the cloud and are accessable as hosted services. Can be found at http://windows.azure.com/

2)SQL Azure Web Portal - Subscription required, data stored in the cloud. Can be found at http://sql.azure.com/

3)Windows Azure AppFabric Web Dev Portal - Subscription required, use Service Bus and Access Control. Found at http://appfabric.azure.com/

4)Windows Azure AppFabric LABS Web Portal - FREE, test environment for developing against the AppFabric Service Bus. Found at https://portal.appfabriclabs.com/