"FAILURE is the FIRST STEP towards SUCCESS"

Documentation: Amazon S3

Written By: Sascha Matzke (sascha.matzke@didolo.com)

Documentation Author: Ankit Singh (ankit@hover.in or ankitsingh.05@gmail.com)

Modules: erlaws.erl, erlaws_s3.erl, erlaws_util.erl, erlaws.hrl

Motive: The erlaws*.* modules are not written by me. Sascha Matzke is the orginal creator of the modules. The motive to write this documentation is to make it easy to use. Due to lack of proper documentation, I struggled for 3 days to make it work. So, I decided to write a proper documentation for the future users and to make there life easy. I hope that I will make justice to the modules through this documentation. If I miss any point then please add to it.

Description:

Erlaws is a collection of client implementations of Amazon's WebServices offerings. Currently there are clients for S3, SQS and SDB written in erlang.

http://code.google.com/p/erlaws/

Build:

Check out the latest code from svn and issue erl -make to build the sources.

How to use erlaws_s3.erl modules?

(ankit@127.0.0.1)1> erlaws:start().

We have to start ceratain services like inets, sasl, httpd, httpc etc for using erlaws_s3.erl. For that we erlaws.erl code is provided

Likely gives output message when successfully started the gen-server:

=PROGRESS REPORT==== 19-Feb-2009::13:11:45 ===

supervisor: {local,sasl_safe_sup}

started: [{pid,<0.625.0>},

{name,alarm_handler},

{mfa,{alarm_handler,start_link,[]}},

{restart_type,permanent},

{shutdown,2000},

{child_type,worker}]

=PROGRESS REPORT==== 19-Feb-2009::13:11:45 ===

supervisor: {local,sasl_safe_sup}

started: [{pid,<0.626.0>},

{name,overload},

{mfa,{overload,start_link,[]}},

{restart_type,permanent},

{shutdown,2000},

{child_type,worker}]

=PROGRESS REPORT==== 19-Feb-2009::13:11:45 ===

supervisor: {local,inets_sup}

started: [{pid,<0.646.0>},

{name,tftp_sup},

{mfa,{tftp_sup,start_link,[[]]}},

{restart_type,permanent},

{shutdown,infinity},

{child_type,supervisor}]

=PROGRESS REPORT==== 19-Feb-2009::13:11:45 ===

application: inets

started_at: 'ankit@127.0.0.1'

erlaws_s3.erl

List of modules in erlaws_s3.erl :

create_bucket/2 create_bucket/3 delete_bucket/2 delete_object/3

get_object/3 info_object/3 instance/3 list_buckets/1

list_contents/2 list_contents/3 module_info/0 module_info/1

new/3 put_object/6

When open this program you'll see a line

-module(erlaws_s3, [AWS_KEY, AWS_SEC_KEY, SECURE]).

Its a parameterized modules. You have to instantiate to get module instance which can be used in a function call just like any module name. So, you have to pass above arguments given. Example

S3 = erlaws_s3:new("2R9TY8PSBSZK15YVE659","qbltwr2uA9uET2XXG5FRgZqSo1yD2jEc+3o0gEzs",false).

AWS_KEY = “2R9TY8PSBSZK15YVE659”

AWS_SEC_KEY = “qbltwr2uA9uET2XXG5FRgZqSo1yD2jEc+3o0gEzs”

SECURE = true/ false (true : httpc & false: http request)

NOTE: if you don't do this then the error will come in every function saying function not defined like list_buckets/1 gives error :

(ankit@127.0.0.1)3> erlaws_s3:list_contents("thumbs.hoverin").

** exception error: undefined function erlaws_s3:list_contents/1

Now, you can call above modules. Some Examples.

erlaws_s3:list_contents("bucket.hoverin",S3).

erlaws_s3:get_object("bucket.hoverin",[],S3). Or erlaws_s3:get_object(“bucket.hoverin”,”as.jpg”,S3).

erlaws_s3:info_object("bucket.hoverin",[],S3). Or

erlaws_s3:info_object(“bucket.hoverin”,”as.jpg”,S3).

For uploading use put_object. Will go through the required attributes with syntax & examples

Sample:

S3:put_object("bucket.hoverin","as.jpg",F,"image/jpeg",[{"x-amz-acl","public-read"}]).

F is the binary file to be sent. I read the files using file in erlang. Ex.

(hover1@0.0.0.0)47> {ok, F} = file:read_file("as.jpg").

{ok,<<255,216,255,224,0,16,74,70,73,70,0,1,1,1,0,72,0,72,>

0,0,255,219,0,67,0,5,3,...>>}

Attributes:

First – bucket name i.e bucket.hoverin

Second - Key name of the file to be given i.e as.jpg

Third - Binary file (example given how to make binary file of jpg or any other file) i.e F

Fourth – Content Type

Fifth – Meta Data or permission to be given to file. i.e [{"x-amz-acl","public-read"}]

Following are canned ACLs that are supported for REST.

• private—Owner gets FULL_CONTROL.

  No one else has access rights (default).

• public-read—Owner gets FULL_CONTROL and the anonymous principal is granted READ access.

  If this policy is used on an object, it can be read from a browser with no authentication.

• public-read-write—Owner gets FULL_CONTROL, the anonymous principal is granted READ and WRITE access.

  This can be a useful policy to apply to a bucket, but is generally not recommended.

• authenticated-read—Owner gets FULL_CONTROL, and any principal authenticated as a registered Amazon S3 user is granted READ access.

For more information about the permissions & headers please refer the link given below :-

http://docs.amazonwebservices.com/AmazonS3/latest/index.html?RESTAccessPolicy.html

NOTE: In line 324 of the erlaws_s3.erl, I modified the code to work accourding to user requirements.

Follow the step:

-Comment this line

%%buildMetadataHeaders(Tail, [{string:to_lower("x-amz-meta-"++Key), Value} | Acc]).

-And paste the following line below commented line.

buildMetadataHeaders(Tail, [{string:to_lower(""++Key), Value} | Acc]).

I removed “x-amz-meta-” as when I run the program, the final header I get

“x-amz-meta-x-amz-acl” instead of only “a-amz-xcl”. Due to this, I was not able to access my files from Amazon S3 through URL intstead of files was present in the bucket.

You can add more meta Headers, if like to. For example,

S3:put_object("bucket.hoverin","as.jpg",F,"image/jpeg",[{“x-amz-meta-username”,”ankit”},{"x-amz-acl","public-read"}]).

Hope you find this document helpful for your Amazon S3 operations.

Have a nice day!