Getting the Amazon Web Service to Work and add Book Cover


#1

The Amazon Web Service now requires two additional things to work:

1- A signature to prove that nobody is trying to temper with your request
2- A timestamp to specify an expiration date for your request

You first need to register to AWS to get your own Access Key and your own Secret Key. Anybody can see your Access Key because it’s in plain text in the HTML request. The secret key is used to encode the whole request and generate a signature. By encoding the request again with the same secret key and comparing the two signatures together, AWS knows if the request it receives comes from you or not.

If I implement a Web Service I will definitely use the same signature process so I thought it was worth investigating how to get it to work.

The string that is encoded is not exactly the html request but a multiline string like:
GET
ecs.amazonaws.com
/onca/xml
AWSAccessKeyId=xxxxxxxxxxxxxxx&Keywords=photography&Operation=ItemSearch&…etc…

I found the function encodeBase64WithNewLines there: cocoadev.com/index.pl?BaseSixtyFour.

Once you’ve signed up to AWS, find the “Account” tab, then go to “Security Credentials”. In the “Access Keys” panel you’ll find your Access Key ID and your Secret Access Key.

All the code below is added to AppController.m

[color=#0040BF]1) Add the libcrypto.dylib Framework in your project[/color]
in XCode, right click on Frameworks, Select Add / Existing Frameworks, restrict to “dylibs” and find “libcrypto.dylib”

[color=#0040BF]2) Add your own AWS credentials[/color]
#define AWS_ID @“xxxxxxxxxxxxxxx”
#define SECRET_KEY @"-----------------------------------------------------------------"

[color=#0040BF]3) Add a method to NSData to do the encoding (Category)[/color]
@interface NSData (Base64)

  • (NSString *)encodeBase64;
  • (NSString *)encodeBase64WithNewlines: (BOOL) encodeWithNewlines;

@end

@implementation NSData (Base64)

  • (NSString *)encodeBase64
    {
    return [self encodeBase64WithNewlines: NO];
    }

  • (NSString *)encodeBase64WithNewlines: (BOOL) encodeWithNewlines
    {
    // Create a memory buffer which will contain the Base64 encoded string
    BIO * mem = BIO_new(BIO_s_mem());

    // Push on a Base64 filter so that writing to the buffer encodes the data
    BIO * b64 = BIO_new(BIO_f_base64());
    if (!encodeWithNewlines)
    BIO_set_flags(b64, BIO_FLAGS_BASE64_NO_NL);
    mem = BIO_push(b64, mem);

    // Encode all the data
    BIO_write(mem, [self bytes], [self length]);
    BIO_flush(mem);

    // Create a new string from the data in the memory buffer
    char * base64Pointer;
    BIO_get_mem_data(mem, &base64Pointer);

// deprecated
// long base64Length = BIO_get_mem_data(mem, &base64Pointer);
// NSString * base64String = [NSString stringWithCString: base64Pointer
// length: base64Length];

BIO_get_mem_data(mem, &base64Pointer);
NSString * base64String = [NSString stringWithUTF8String: base64Pointer];

// Clean up and go home
BIO_free_all(mem);
return base64String;

}

@end

[color=#0040BF]4) Add a method to NSString to encode the URL properly (Category)[/color]
@interface NSString (ForURL)

  • (NSString*)reallyEncodeURL;

@end

@implementation NSString (ForURL)

  • (NSString*)reallyEncodeURL
    {
    return (NSString*)CFURLCreateStringByAddingPercentEscapes(
    NULL,
    (CFStringRef)self,
    NULL,
    (CFStringRef)@"!*’();:@&=+$,/?%#[]",
    kCFStringEncodingUTF8 );
    }

@end

[color=#0040BF]5) In the method fetchBooks, replace the creation of the urlString with the following code[/color]

    ...
    NSLog(@"searchString = %@", searchString);

//Timestamp format : needs to be yyyy-mm-ddThh:mm+/-timediff , it's also set to tomorrow to avoid Request Expired error.
NSDate *now = [[NSDate alloc] initWithTimeIntervalSinceNow:24 * 60 * 60];
NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
[dateFormatter setDateFormat:@"yyyy-MM-dd'T'hh:mm:ssZ"];
NSString *timestamp = [dateFormatter stringFromDate:now];
NSString * encoded_timestamp = [timestamp reallyEncodeURL];
NSLog(@"Date formatted %@ => %@", timestamp, encoded_timestamp);

// parameters have to be binary sorted
NSString *service_URL = @"ecs.amazonaws.com";
NSString *service_path = @"/onca/xml";
NSString *access_ID = [NSString stringWithFormat:@"AWSAccessKeyId=%@&",AWS_ID];
NSString *keywords = [NSString stringWithFormat:@"Keywords=%@&",searchString];
NSString *operation = @"Operation=ItemSearch&";
NSString *response_group = @"ResponseGroup=ItemAttributes%2CImages&";
NSString *search_index = @"SearchIndex=Books&";
NSString *service = @"Service=AWSEcommerceService&";
NSString *signature_method = @"SignatureMethod=HmacSHA256&";
NSString *signature_version = @"SignatureVersion=2&";
NSString *ts = [NSString stringWithFormat:@"Timestamp=%@&", encoded_timestamp];
NSString *version = @"Version=2010-01-01";

//Signature
NSString *string_to_sign = [NSString stringWithFormat:@"GET\n%@\n%@\n%@%@%@%@%@%@%@%@%@%@", 
								service_URL, service_path, access_ID, keywords, operation, response_group, 
								search_index, service, signature_method, signature_version, ts, version];

NSLog(@"String to sign = %@", string_to_sign);

NSString *key = SECRET_KEY;
NSString *data = string_to_sign;

const char *cKey  = [key cStringUsingEncoding:NSASCIIStringEncoding];
const char *cData = [data cStringUsingEncoding:NSASCIIStringEncoding];

unsigned char cHMAC[CC_SHA256_DIGEST_LENGTH];

CCHmac(kCCHmacAlgSHA256, cKey, strlen(cKey), cData, strlen(cData), cHMAC);

NSData *HMAC = [[NSData alloc] initWithBytes:cHMAC
									  length:sizeof(cHMAC)];

NSString *hash = [HMAC encodeBase64];
NSString *encoded_hash = [hash reallyEncodeURL];


NSLog(@"hash = %@- => %@", hash, encoded_hash);

// Create the URL
NSString *urlString = [NSString stringWithFormat:@"http://%@%@?%@%@%@%@%@%@%@%@%@%@&Signature=%@", 
							service_URL, service_path, access_ID, keywords, operation, response_group, 
							search_index, service, signature_method, signature_version, ts, version, encoded_hash];

NSLog(@"urlString = %@", urlString); // you can paste this in your web browser to see the XML returned

    NSURL *url = [NSURL URLWithString:urlString];
    ....

[color=#0040BF]6) Add the image next to the title[/color]
If you look at the parameter “ResponseGroup”, you’ll see that I added “Images”. AWS will then return the URL to various image sizes of the book cover

Open MainMenu.xib, add a new column to the TableView and drag an Image Cell into it. Name the column “Image”.

Change the objectValueForTableColumn method as shown below:

  • (id)tableView:(NSTableView *)tv objectValueForTableColumn:(NSTableColumn *)tableColumn row:(int)row
    {
    NSXMLNode *node = [itemNodes objectAtIndex:row];
    NSString *xPath = [tableColumn identifier];

    if ([[tableColumn identifier] isEqual:@“Image”])
    {
    NSString *imageXPath = @“SmallImage/URL”;
    NSString *urlString = [self stringForPath:imageXPath ofNode:node];
    if (urlString) {
    NSURL *url = [NSURL URLWithString:urlString];

      	NSImage *image = [[NSImage alloc] initWithContentsOfURL:url];
      	return image;
      } else {
      	return nil;
      }
    

    }

    return [self stringForPath:xPath ofNode:node];
    }

Et Voila!!

That was a good challenge !!

Thanks Aaron for a great book!


#2

Thanks for posting this clear and complete write-up. You rule!


#3

Last friday I was thinking by myself that I should post my solution on here, now duzmac has beaten me. Anyway, it is funny to see that your solution matches mine for the greater part. I used the same base64 method as well as the one for the URL encoding. Even though I’ve tried openssl (and possibly another framework as well) I ended up using libcrypto too. It did take me a long time to get the signing sorted though; even when I used the sample string from Amazon I wasn’t able to generate the same output. I’m afraid my main problem was converting the unsigned char back to a NSString, using NSData did eventually the job for me.

I put all arguments of the request in a dictionary, so it can be sorted and used again with different arguments in other parts of the application. This part goes into fetchBooks:sender

[code]// Add variable keywords to Dictionary
NSMutableDictionary *reqData = [NSMutableDictionary dictionary];

[reqData setObject:[searchField stringValue] forKey:@“Keywords”];
[reqData setObject:@“ItemSearch” forKey:@“Operation”];
[reqData setObject:@“Books” forKey:@“SearchIndex”];

NSURLRequest *urlRequest;
urlRequest = [self generateAmazonRequest:reqData];[/code]

The method below takes the (partial) dictionary and adds everything that is required. After that, the items in the dictionary will be sorted and put in an escaped NSString. The NSString is signed and in the end the NSURLRequest is returned. I added my separate methods to generate the signature and to escape the url string as well. The encodeBase64WithNewlines method I used is exactly the same as the one duzmac had found.

[code]- (NSURLRequest *)generateAmazonRequest:(NSMutableDictionary *)dict
{
// Complete dictionary
if ([dict objectForKey:@“AWSAccessKeyId”] == nil) {
[dict setObject:AWS_ID forKey:@“AWSAccessKeyId”];
}
if ([dict objectForKey:@“Responsegroup”] == nil) {
[dict setObject:@“Medium” forKey:@“Responsegroup”];
}
if ([dict objectForKey:@“Service”] == nil) {
[dict setObject:@“AWSECommerceService” forKey:@“Service”];
}
if ([dict objectForKey:@“Version”] == nil) {
[dict setObject:@“2009-10-01” forKey:@“Version”];
}

// Make timestamp
NSTimeZone *gmtZone = [NSTimeZone timeZoneWithName:@"GMT"];
NSString *dateString = [[NSDate date] descriptionWithCalendarFormat:@"%Y-%m-%dT%H:%M:%SZ" timeZone:gmtZone locale:nil];
[dict setObject:dateString forKey:@"Timestamp"];

// Sort keys by name and make the URL-string
NSArray *myKeys = [dict allKeys];
NSArray *sortedKeys = [myKeys sortedArrayUsingSelector:@selector(compare:)];

NSString *value;
NSString *urlString;
NSString *stringToSign;
urlString = @"";

for (NSString *key in sortedKeys) {
	value = [self escapeUrl:[dict objectForKey:key]];
	urlString = [NSString stringWithFormat:@"%@%@=%@&", urlString, key, value];
}

// Strip off unnecessary trailing &
NSRange range = NSMakeRange(0, [urlString length] - 1);
urlString = [urlString substringWithRange:range];

// Get signature and add it to the url
stringToSign = [NSString stringWithFormat:@"GET\necs.amazonaws.com\n/onca/xml\n%@", urlString];
NSString *signedString;
signedString = [self generateHMACSign:stringToSign];
signedString = [self escapeUrl:signedString];
signedString = [NSString stringWithFormat:@"&Signature=%@", signedString];

urlString = [urlString stringByAppendingString:signedString];
urlString = [@"http://ecs.amazonaws.com/onca/xml?" stringByAppendingString:urlString];

NSURL *url = [NSURL URLWithString:urlString];
NSURLRequest *urlRequest = [NSURLRequest requestWithURL:url
									cachePolicy:NSURLRequestReturnCacheDataElseLoad 
 								timeoutInterval:30];
return urlRequest;

}

  • (NSString *)generateHMACSign:(NSString *)stringToSign
    {
    // Declare and init secret
    const char *secret;
    secret = (const char *)[AWS_SECRET cStringUsingEncoding:NSASCIIStringEncoding];
    long secretLen;
    secretLen = strlen(secret);

    // Get input as NSData
    NSData *input;
    input = [stringToSign dataUsingEncoding:NSASCIIStringEncoding];

    // Declare/init output pointer
    unsigned char *output;
    output = malloc(CC_SHA256_DIGEST_LENGTH);

    // Common Crypto operations (get HMAC-SHA256 signature)
    CCHmacContext hctx;
    CCHmacInit(&hctx, kCCHmacAlgSHA256, secret, secretLen);
    CCHmacUpdate(&hctx, [input bytes], [input length]);
    CCHmacFinal(&hctx, output);

    // Convert to base64
    NSString *baseString;
    NSData *convert = [NSData dataWithBytes:output length:CC_SHA256_DIGEST_LENGTH];
    baseString = [convert encodeBase64WithNewlines:NO];

    return baseString;
    }

  • (NSString *)escapeUrl:(NSString *)input
    {
    return (NSString )CFURLCreateStringByAddingPercentEscapes(NULL,
    (CFStringRef)input,
    NULL,
    (CFStringRef)@"!
    ’();:@&=+$,/?%#[]",
    kCFStringEncodingUTF8 );
    }[/code]


#4

Thanks duzmac for this great post.

For anyone who is having trouble getting this fix working (as I did), there are a couple of header files you must import that are not mentioned above. Others may work, but I was able to get it to work by doing the following:

Follow all of duzmac’s steps above, then at the top of AppController.m add:

#import <openssl/ssl.h> //for BIO, etc #import <CommonCrypto/CommonHMAC.h> //for kCCHmacAlgSHA256 #import <CommonCrypto/CommonDigest.h> //for CC_SHA256_DIGEST_LENGTH


#5

Really helpful, and it runs for me, but I’m getting an error from Amazon telling me the Signature they calculate isn’t the same as the one I send. AWS_ID and private key are identical to those reported on my account. My code is identical to the code duzmac kindly posted - after many hours of staring at it… but getting an information note that BIO_flush(mem) output is unused.

Should the BIO_get_mem_data(mem, &base64Pointer) line really be duplicated? (I’ve tried it both ways with no difference of outcome).

char * base64Pointer;
[color=#FF0000]BIO_get_mem_data(mem, &base64Pointer);[/color]

// deprecated
// long base64Length = BIO_get_mem_data(mem, &base64Pointer);
// NSString * base64String = [NSString stringWithCString: base64Pointer
// length: base64Length];

[color=#FF0000]BIO_get_mem_data(mem, &base64Pointer);[/color]

Getting really frustrated - any ideas would be most welcome, please…


#6

I used this code and worked the first time

After a few tests, returns that my account/ID is not valid.

If I restart the app/quit, works fine again. But only for 1 or 2 searchs. And then again returns invalid.

Any help?