< in July 2016 and won a top **C++ project of the month** award.
Introduction {#intro}
------------
This article will walk through the process of using [gSOAP](products.html) to connect with [Amazon Web Services' Simple Storage Service (S3)](https://aws.amazon.com/s3/). We will create a bucket and then upload an object to the bucket using a streaming DIME attachment. See [Amazon S3 Web Services example](https://www.genivia.com/examples/aws/index.html) for downloadable source code and build steps.
gSOAP is a toolkit that auto-generates SOAP and XML data binding for C and C++. It simplifies the development of SOAP/XML web services, and in our case, the consumption of existing Web services.
Amazon S3 is a web service that is used to store files. They offer a SOAP API, so we can easily use the gSOAP toolkit to generate code to interact with AWS S3.
Installation and Setup {#setup}
----------------------
### gSOAP
Refer to the relevant section on [this page](downloads.html) for instructions on how to install gSOAP on your system.
### OpenSSL
You will need OpenSSL installed because Amazon requires all SOAP requests to be encrypted. You can read more about OpenSSL and how to install it at [their website](https://www.openssl.org).
### AWS, Access Keys, and Config File
An [AWS S3](http://aws.amazon.com/s3) account is necessary in order to use the service. You'll also need to [create access keys](https://aws.amazon.com/developers/access-keys) to send any requests to the AWS API. Keep these keys close by.
It's **extremely recommended** to use a config file to safely store your access keys. This way, you can read in the keys when necessary and you won't accidentally upload them to any version control systems. To generate a config file, we can use an AWS tool called the [AWS Command Line Interface](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html). Install it and run:
[command]
aws configure
Follow the prompts to enter your information. This tool will create a local file named *credentials* in a folder named *.aws* in your home directory (environment variable `~` on Unix systems or `%USERPROFILE%` on Windows systems). For more help on using the AWS CLI tool to create a local config file, refer to [this page](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html).
If you don't want to use the AWS CLI, you could also manually make a config file. It's best to store it in a directory not associated with any version control. Alternatively, you could use environment variables to store your access keys. If you choose one of these options, you'll have to change the parser in the [Reading Keys from Config File](#keys) section, as it assumes that the file has the format generated by the AWS CLI.
Generate C++ Data-bindings from the AWS S3 WSDL with gSOAP {#wsdl}
----------------------------------------------------------
It's recommended to make a new, empty directory for the files we are about to generate.
**NOTE**: In at least two cases (`CreateBucket` and `CopyObject`), Amazon's SOAP responses don't match their promised schema (this will hopefully be corrected soon). Because of this, it is necessary to edit the *typemap.dat* file to edit gSOAP's generated classes to ensure we can access every response from Amazon.
I recommended first copying the *typemap.dat* file from the *gsoap/* directory into your new directory, and then add the following lines to it (anywhere):
_s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
_s3__CopyObjectResponse = $ s3__CopyObjectResult* CopyObjectResponse;
Now, we will generate the C++ data bindings.
### Step 1
Use gSOAP's `wsdl2h` tool on Amazon's S3 WSDL to generate a header file (we'll name it *aws-s3.h*).
[command]
wsdl2h -t typemap.dat -o aws-s3.h http://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl
The generated *aws-s3.h* header file contains all of the class declarations for Amazon S3's service operation requests and responses, along with other Amazon S3 functions and gSOAP functions.
### Step 2
The second step in gSOAP's code generation is to use the `soapcpp2` tool on the header file created from the wsdl2h tool.
[command]
soapcpp2 -C -r -j aws-s3.h
This generates client-side code (`-C` option), a report (`-r` option) with C++ service proxies and objects (`-j` option) from the *aws-s3.h* header:
* soapStub.h a copy of the specification in plain C/C++ header file syntax without annotations.
* soapH.h declares XML serializers.
* soapC.c implements XML serializers.
* soapAmazonS3SoapBindingProxy.h defines the client-side XML services API proxy class `AmazonS3SoapBindingProxy`.
* soapAmazonS3SoapBindingProxy.cpp implements the client-side XML services API proxy class `AmazonS3SoapBindingProxy`.
* AmazonS3SoapBinding.nsmap XML namespace binding table, you should #include this file.
* soapReadme.md service and data binding interface details.
Check out the auto-generated [report](examples/aws/soapReadme.html) on the *aws-s3.h* header for a list of classes and types.
For more general information on how gSOAP works, visit the [Developer Center](dev.html) or the [User Guide](doc/guide/html/index.html).
Access Keys and Signature for AWS S3 {#keys}
------------------------------------
### Reading Keys from Config File
In the [Installation and Setup](#setup) section, we stored our access keys in a file called *credentials* in the *.aws* folder of your home directory. We'll define this function to read in the keys from that file.
// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
std::ifstream credentialsFile(path.c_str());
if (!credentialsFile.is_open())
return false;
std::string line;
while (std::getline(credentialsFile, line)) {
// Keep going until we get to the desired user
if (line.find(user) == std::string::npos)
continue;
while (std::getline(credentialsFile, line)) {
// Keep going until we get to the access key lines
if (line.find("aws_access_key_id") == std::string::npos)
continue;
// Grab keys and trim whitespace
size_t first, last;
accessKey = line.substr(line.find_first_of('=')+1);
first = accessKey.find_first_not_of(' ');
if (first == std::string::npos)
return false;
last = accessKey.find_last_not_of(' ');
accessKey.substr(first, last-first+1).swap(accessKey);
std::getline(credentialsFile, line);
secretKey = line.substr(line.find_first_of('=')+1);
first = secretKey.find_first_not_of(' ');
if (first == std::string::npos)
return false;
last = secretKey.find_last_not_of(' ');
secretKey.substr(first, last-first+1).swap(secretKey);
return true;
}
}
return false;
}
We can call this function to set local variables, keeping hard-coded access keys out of our source.
// Load AWS keys from file
std::string accessKey, secretKey;
std::string credentialsFile = "path_to_aws_credentials_file";
std::string user = "default";
if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
std::cout << "Couldn't read AWS keys for user " << user
<< " from file " << credentialsFile << '\n';
return 0;
}
You must supply the path to the file containing your access keys. If you didn't use the `--profile` option when running `aws configure`, you can leave the `user` variable as `default`.
### How to Construct a Signature
Amazon needs a specially-constructed signature for every SOAP request in order to authenticate and process the request. We will define the function `soap_make_s3__signature` in order to make creating a signature painless. Its inputs are a soap context, an operation name (like "`CreateBucket`"), and your AWS secret key. It returns a string with the base64-encoded version of the HMAC-SHA1 hashed string "`AmazonS3`" + `OPERATION_NAME` + `Timestamp`.
// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
std::string signature = "AmazonS3";
signature += operation;
char UTCstamp[40]; //to hold ISO 8601 time format
time_t now;
time(&now);
strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
signature += UTCstamp;
// Get the HMAC-SHA1 digest of the signature string
unsigned char * digest;
digest = HMAC(EVP_sha1(), key, strlen(key),
(unsigned char*)(signature.c_str()),
signature.length(), NULL, NULL);
char signatureBase64[20];
// Convert the digest to base64
soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
return std::string(signatureBase64);
}
The functions `HMAC` and `EVP_sha1` are from the OpenSSL libraries (which we'll link with in the [Compiling](#compiling) section).
So we can simply call our new function and supply the necessary parameters...
std::string signature = soap_make_s3__signature(aws.soap, "OperationName", secretKey.c_str());
...making it easy to create signatures whenever we need to make an API request. Note that we're using the `secretKey` variable that we created in the previous section, rather than using a hard-coded string.
Using gSOAP to Connect with AWS S3 {#connect}
----------------------------------
**Note**: In these examples, we are going to use classes (`AmazonS3SoapBindingProxy`, `_s3__CreateBucket`, etc.) we generated with gSOAP's wsdl2h and soapcpp2 tools in the previous step. Again, if you are curious what generated classes are available or what member data they have, check the generated *aws-s3.h* file or the auto-generated [report](examples/aws/soapReadme.html) on it.
First Example: Creating a Bucket {#example1}
--------------------------------
For the first example, let's create a new bucket.
For gSOAP's generated code to work, we need to include the generated header file and namespace mapping file.
#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"
We're also going to add a convenient template function to allocate primitive values on gSOAP's managed heap using `soap_malloc()` with context `soap`. This isn't necessary, but will make the code cleaner and easier to manage.
// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
T *p = (T*)soap_malloc(soap, sizeof(T));
*p = val;
return p;
}
Now we are going to write the code to connect to S3. This is all going to be in our `main` function. We're going to create a proxy of auto-generated type `AmazonS3SoapBindingProxy` to invoke AWS S3 services. Then we'll create a `CreateBucket` request of auto-generated type `_s3__CreateBucket` and set its arguments.
// Create a proxy to invoke AWS S3 services
AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);
// Set the arguments of the CreateBucket service
_s3__CreateBucket createBucketReq;
std::string bucketName = "BucketName";
createBucketReq.Bucket = bucketName;
createBucketReq.AWSAccessKeyId = soap_new_std__string(aws.soap);
*createBucketReq.AWSAccessKeyId = accessKey;
createBucketReq.Timestamp = soap_make(aws.soap, time(0));
createBucketReq.Signature = soap_new_std__string(aws.soap);
*createBucketReq.Signature = soap_make_s3__signature(aws.soap, "CreateBucket", secretKey.c_str());
Make sure to change the `bucketName string` to your desired bucket name. For the `Signature`, we use the function we made in the [How to Construct a Signature](#signature) section of this article, with the operation name `"CreateBucket"`.
We're going to need to catch the response of our request, so we'll make an instance of the auto-generated class `_s3__CreateBucketResponse`.
// To store the result of the service
_s3__CreateBucketResponse createBucketRes;
Now, we are going to actually call the service.
// Create a bucket
if (aws.CreateBucket(&createBucketReq, createBucketRes)) {
aws.soap_stream_fault(std::cerr);
}
/*
NOTE: you must add the line:
_s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
to the typemap.dat file because Amazon's response doesn't match
their promised schema. This adds the variable CreateBucketResponse
to the _s3__CreateBucketResponse class so we can access the response.
*/
else if (createBucketRes.CreateBucketResponse) {
s3__CreateBucketResult &result = *createBucketRes.CreateBucketResponse;
std::cout << "You are the owner of bucket'" <<
result.BucketName << "'." << std::endl;
}
This will send `createBucketReq` to Amazon S3 (in proper SOAP format, thanks to gSOAP) and put the response in `createBucketRes`. You can see the new bucket in your [AWS Management Console](https://aws.amazon.com/console/).
The comment above the success block is just a reminder of what was said in the note at the top of the [Generate C++ data-binding from the AWS S3 WSDL](#wsdl) section of this article.
And when we're all finished, make sure to call the proxy's `destroy` function.
// Delete all managed data
aws.destroy();
And that's all we need to do to create a bucket. The final *.cpp* file (we'll call it *createbucket.cpp*) will look like this:
/*
createbucket.cpp
Example AWS S3 CreateBucket service invocation
*/
#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"
#include <fstream>
// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
T *p = (T*)soap_malloc(soap, sizeof(T));
*p = val;
return p;
}
// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
std::string signature = "AmazonS3";
signature += operation;
char UTCstamp[40]; //to hold ISO 8601 time format
time_t now;
time(&now);
strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
signature += UTCstamp;
// Get the HMAC-SHA1 digest of the signature string
unsigned char * digest;
digest = HMAC(EVP_sha1(), key, strlen(key),
(unsigned char*)(signature.c_str()),
signature.length(), NULL, NULL);
char signatureBase64[20];
// Convert the digest to base64
soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
return std::string(signatureBase64);
}
// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
std::ifstream credentialsFile(path.c_str());
if (!credentialsFile.is_open())
return false;
std::string line;
while (std::getline(credentialsFile, line)) {
// Keep going until we get to the desired user
if (line.find(user) == std::string::npos)
continue;
while (std::getline(credentialsFile, line)) {
// Keep going until we get to the access key lines
if (line.find("aws_access_key_id") == std::string::npos)
continue;
// Grab keys and trim whitespace
size_t first, last;
accessKey = line.substr(line.find_first_of('=')+1);
first = accessKey.find_first_not_of(' ');
if (first == std::string::npos)
return false;
last = accessKey.find_last_not_of(' ');
accessKey.substr(first, last-first+1).swap(accessKey);
std::getline(credentialsFile, line);
secretKey = line.substr(line.find_first_of('=')+1);
first = secretKey.find_first_not_of(' ');
if (first == std::string::npos)
return false;
last = secretKey.find_last_not_of(' ');
secretKey.substr(first, last-first+1).swap(secretKey);
return true;
}
}
return false;
}
int main(int argc, char **argv) {
// Load AWS keys from file
std::string accessKey, secretKey;
// Use the path to your AWS credentials file
std::string credentialsFile = (argc > 2 ? argv[2] : "path_to_aws_credentials_file");
std::string user = "default";
if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
std::cout << "Couldn't read AWS keys for user " << user
<< " from file " << credentialsFile << '\n';
return 0;
}
// Create a proxy to invoke AWS S3 services
AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);
// Create bucket
// Set the arguments of the CreateBucket service operation
_s3__CreateBucket createBucketReq;
std::string bucketName = (argc > 1 ? argv[1] : "BucketName");
createBucketReq.Bucket = bucketName;
createBucketReq.AWSAccessKeyId = soap_new_std__string(aws.soap);
*createBucketReq.AWSAccessKeyId = accessKey;
createBucketReq.Timestamp = soap_make(aws.soap, time(0));
createBucketReq.Signature = soap_new_std__string(aws.soap);
*createBucketReq.Signature = soap_make_s3__signature(aws.soap, "CreateBucket", secretKey.c_str());
// Store the result of the service
_s3__CreateBucketResponse createBucketRes;
// Create a bucket
if (aws.CreateBucket(&createBucketReq, createBucketRes)) {
aws.soap_stream_fault(std::cerr);
}
/*
NOTE: you must add the line:
_s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
to the typemap.dat file because Amazon's response doesn't match
their promised schema. This adds the variable CreateBucketResponse
to the _s3__CreateBucketResponse class so we can access the response.
*/
else if (createBucketRes.CreateBucketResponse) {
s3__CreateBucketResult &result = *createBucketRes.CreateBucketResponse;
std::cout << "You are the owner of bucket '" << result.BucketName << "'." << std::endl;
}
// Delete all managed data
aws.destroy();
return 0;
}
Compiling the CreateBucket Example {#compiling}
----------------------------------
To compile our program, we must use OpenSSL because the AWS S3 service requires HTTPS to ensure that the credentials are securely transmitted. This means we must link with the OpenSSL and OpenSSL crypto libraries. The command looks like:
[command]
c++ -DWITH_OPENSSL -o createbucket createbucket.cpp \
soapAmazonS3SoapBindingProxy.cpp soapC.cpp stdsoap2.cpp -lssl -lcrypto
You may have to fix the path to *stdsoap2.cpp* to be an absolute path to the *stdsoap2.cpp* file in your *gsoap/* directory from the installation. You may also have to include a `-I` option with the path to the *gsoap/import* folder.
Compile the code, run `createbucket`, and a new bucket will be created.
In the final *.cpp* file, note that we check the command line arguments (`argv`) when setting `credentialsFile` and `bucketName`. This allows the program to be called with arguments:
[command]
./createbucket BucketName path_to_credentials_file
For security reasons, the default limit on DIME attachment size received is 8 MB. To increase the limit, compile with `-DSOAP_MAXDIMESIZE=nnn`, where `nnn` is the size in bytes. For example:
[command]
c++ -DSOAP_MAXDIMESIZE=104857600 -DWITH_OPENSSL -o createbucket createbucket.cpp \
soapAmazonS3SoapBindingProxy.cpp soapC.cpp stdsoap2.cpp -lssl -lcrypto
This sets the maximum size of DIME attachments that can be received to 100 MB.
Second Example: Uploading a File with PutObject Using a DIME Attachment with Streaming {#example2}
--------------------------------------------------------------------------------------
Now that we have created a bucket, let's upload a file into it.
AWS S3's SOAP API has two different operations for uploading objects: `PutObjectInline` and `PutObject`. The inline version expects the data directly included in the body of the SOAP message, while the non-inline version expects the data provided as a DIME attachment. Files over 1MB cannot be uploaded with the inline version, so this example will demonstrate how to use DIME attachments. An example (*putobjectinline.cpp*) for the `PutObjectInline` operation has also been provided in the *aws-s3-gsoap-examples.zip* file.
gSOAP supports [DIME attachments](https://en.wikipedia.org/wiki/Direct_Internet_Message_Encapsulation) with or without streaming. Without streaming, the full file is stored and retrieved in memory, which can be a problem with larger files/smaller devices. We will show how to use gSOAP's DIME attachments **with** streaming, although an example (*putobject.cpp*) without streaming can be found in the *aws-s3-gsoap-examples.zip* file.
We will use the same helper functions (`soap_make`, `getAWSKeys`, and `soap_make_s3__signature`) as we did in the `CreateBucket` example.
Our request will be of auto-generated class `_s3__PutObject`. It requires a bucket name, a key (to name the object we're going to upload), an access key, a timestamp, and a signature. We can also supply name-value metadata pairs (of type `s3__MetadataEntry`) to the `Metadata` container element, to be stored with the object, but this is optional.
// Create a proxy to invoke AWS S3 services
AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);
// Set the arguments of the PutObject service operation
_s3__PutObject putObjectReq;
std::string bucketName = "BucketName";
std::string keyName = "KeyName";
putObjectReq.Bucket = bucketName;
putObjectReq.Key = keyName;
putObjectReq.AWSAccessKeyId = soap_new_std__string(aws.soap);
*putObjectReq.AWSAccessKeyId = accessKey;
putObjectReq.Timestamp = soap_make(aws.soap, time(0));
putObjectReq.Signature = soap_new_std__string(aws.soap);
*putObjectReq.Signature = soap_make_s3__signature(aws.soap, "PutObject", secretKey.c_str());
s3__MetadataEntry *metadataEntry = soap_new_req_s3__MetadataEntry(aws.soap, "Metadata Name", "Metadata Value");
putObjectReq.Metadata.push_back(metadataEntry);
All that's left is to include the data we want to upload as a DIME attachment, encoded in base 64. Streaming DIME attachments with gSOAP is simple enough, but as a reminder, there is a non-streaming version included in the examples *.zip* folder.
An in-depth explanation of the callback functions needed to stream DIME attachments can be read in [this section](doc/guide/html/index.html#DIMEstreaming) of the gSOAP User Guide, but this example will show an implementation. `dime_read_open` will simply return the `handle` (we'll open the file ourselves beforehand), `dime_read` will read in up to `len` bytes from the file and store it in a buffer managed by gSOAP, and `dime_read_close` will close the file.
void *dime_read_open(struct soap *soap, void *handle, const char *id, const char *type, const char *options) {
return handle;
}
size_t dime_read(struct soap *soap, void *handle, char* buf, size_t len) {
return fread(buf, 1, len, (FILE*)handle);
}
void dime_read_close(struct soap *soap, void *handle) {
fclose((FILE*)handle);
}
Great. We need an `xsd__base64Binary` object to facilitate the streaming...
xsd__base64Binary *data = soap_new_xsd__base64Binary(aws.soap);
...and a `FILE` pointer to our file to pass as the `handle` between the callback functions:
FILE *fd = fopen("filename.jpg"), "rb");
if (fd == NULL) return 0;
We can only stream the data if we can get the size of the file (for which we will use `fstat`). If we know the size, then we set the proxy's `soap` struct's callback functions to the ones we just defined and assign the `data` object's `__ptr` and `__size` fields. These values will be used in the gSOAP's streaming callbacks that we defined earlier (the `__ptr` field is sent as the `handle` parameter of the `dime_read_open` function, and the `__size` field is used by the callback functions to know when they've finished).
struct stat sb;
if (!fstat(fileno(fd), &sb) && sb.st_size > 0) {
// We can get the length of the file, so we can stream it
// Set soap callbacks
aws.soap->fdimereadopen = dime_read_open;
aws.soap->fdimereadclose = dime_read_close;
aws.soap->fdimeread = dime_read;
//__ptr must be non-NULL. This is the handle in the callbacks
data->__ptr = (unsigned char*)fd;
data->__size = sb.st_size;
}
If we can't read the size of the file, we need a backup plan. gSOAP [supports HTTP chunking](doc/guide/html/index.html#chunked) (which doesn't require a pre-known size), but we can't use that method as the `PutObject` API call expects a `ContentLength` element. So we'll naively read and set the data in `data->__ptr` until we reach the end, cutting off at an arbitrary size limit.
else {
// We don't know the size, so buffer it
int i;
data->__ptr = (unsigned char*)soap_malloc(aws.soap, MAX_FILE_SIZE);
for (i = 0; i < MAX_FILE_SIZE; i++) {
int c;
if ((c = fgetc(fd)) == EOF)
break;
data->__ptr[i] = c;
}
fclose(fd);
data->__size = i;
}
Be sure to define `MAX_FILE_SIZE`.
#define MAX_FILE_SIZE (1000000) // (1MB). Max size for when we must attach as non-streaming
Great! Now, to tell gSOAP to put the `data` object in the request as a DIME attachment, we need to set either its `type` or `id` field. gSOAP will know to stream the data because we set the callback functions earlier. We also need to call the gSOAP functions `soap_set_dime` with our `soap` context and `soap_set_dime_attachment` with the information we just constructed. Then we'll set the `ContentLength` argument of the request.
// Set DIME attachment with options
char type[] = "text/plain";
data->type = type;
soap_set_dime(aws.soap);
if (soap_set_dime_attachment(aws.soap, (char*)data->__ptr, data->__size, data->type, data->id, 0, data->options)) {
aws.soap_stream_fault(std::cerr);
return 0;
}
putObjectReq.ContentLength = data->__size;
Make sure to set `data->type` appropriately. You can also set `data->id`, but gSOAP will automatically create one if not. `data->options` controls the DIME-specific options field, which can be left blank or set with `soap_dime_option`...
data->options = soap_dime_option(aws.soap, 0, "Optional text");
Now the request is fully constructed! Now we just have to call the API and catch the response, similar to the `CreateBucket` example.
// Store the result of the service
_s3__PutObjectResponse putObjectRes;
// Put object
if (aws.PutObject(&putObjectReq, putObjectRes)) {
aws.soap_stream_fault(std::cerr);
}
else if (putObjectRes.PutObjectResponse) {
s3__PutObjectResult &result = *putObjectRes.PutObjectResponse;
std::cout << "Object with key '" << putObjectReq.Key
<< "' put into bucket '" << putObjectReq.Bucket << "'" << std::endl;
std::cout << "\tEtag: " << result.ETag << std::endl;
std::cout << "\tLast modified: " << soap_dateTime2s(aws.soap, result.LastModified) << std::endl;
}
Again, be sure to call `aws.destroy()` before ending the program.
The final *.cpp* file (we'll call it *putobjectstreaming.cpp*) will look like this:
/*
putobjectstreaming.cpp
Example AWS S3 PutObject service invocation
with DIME attachment (streaming)
*/
#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"
#include <fstream>
#include <sys/stat.h>
#define MAX_FILE_SIZE (1000000) // (1MB). Max size for when we must attach as non-streaming
// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
T *p = (T*)soap_malloc(soap, sizeof(T));
*p = val;
return p;
}
// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
std::string signature = "AmazonS3";
signature += operation;
char UTCstamp[40]; //to hold ISO 8601 time format
time_t now;
time(&now);
strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
signature += UTCstamp;
// Get the HMAC-SHA1 digest of the signature string
unsigned char * digest;
digest = HMAC(EVP_sha1(), key, strlen(key),
(unsigned char*)(signature.c_str()),
signature.length(), NULL, NULL);
char signatureBase64[20];
// Convert the digest to base64
soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
return std::string(signatureBase64);
}
// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
std::ifstream credentialsFile(path.c_str());
if (!credentialsFile.is_open())
return false;
std::string line;
while (std::getline(credentialsFile, line)) {
// Keep going until we get to the desired user
if (line.find(user) == std::string::npos)
continue;
while (std::getline(credentialsFile, line)) {
// Keep going until we get to the access key lines
if (line.find("aws_access_key_id") == std::string::npos)
continue;
// Grab keys and trim whitespace
size_t first, last;
accessKey = line.substr(line.find_first_of('=')+1);
first = accessKey.find_first_not_of(' ');
if (first == std::string::npos)
return false;
last = accessKey.find_last_not_of(' ');
accessKey.substr(first, last-first+1).swap(accessKey);
std::getline(credentialsFile, line);
secretKey = line.substr(line.find_first_of('=')+1);
first = secretKey.find_first_not_of(' ');
if (first == std::string::npos)
return false;
last = secretKey.find_last_not_of(' ');
secretKey.substr(first, last-first+1).swap(secretKey);
return true;
}
}
return false;
}
void *dime_read_open(struct soap *soap, void *handle, const char *id, const char *type, const char *options) {
return handle;
}
size_t dime_read(struct soap *soap, void *handle, char* buf, size_t len) {
return fread(buf, 1, len, (FILE*)handle);
}
void dime_read_close(struct soap *soap, void *handle) {
fclose((FILE*)handle);
}
int main(int argc, char **argv) {
// Load AWS keys from file
std::string accessKey, secretKey;
// Use the path to your AWS credentials file
std::string credentialsFile = (argc > 4 ? argv[4] : "path_to_aws_credentials_file");
std::string user = "default";
if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
std::cout << "Couldn't read AWS keys for user " << user
<< " from file " << credentialsFile << '\n';
return 0;
}
// Create a proxy to invoke AWS S3 services
AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);
// Put object in bucket (streaming)
// Set the arguments of the PutObject service operation
_s3__PutObject putObjectReq;
std::string bucketName = (argc > 1 ? argv[1] : "BucketName");
std::string keyName = (argc > 2 ? argv[2] : "KeyName");
putObjectReq.Bucket = bucketName;
putObjectReq.Key = keyName;
putObjectReq.AWSAccessKeyId = soap_new_std__string(aws.soap);
*putObjectReq.AWSAccessKeyId = accessKey;
putObjectReq.Timestamp = soap_make(aws.soap, time(0));
putObjectReq.Signature = soap_new_std__string(aws.soap);
*putObjectReq.Signature = soap_make_s3__signature(aws.soap, "PutObject", secretKey.c_str());
s3__MetadataEntry *metadataEntry = soap_new_req_s3__MetadataEntry(aws.soap, "Metadata Name", "Metadata Value");
putObjectReq.Metadata.push_back(metadataEntry);
// Set up gSOAP DIME streaming
xsd__base64Binary *data = soap_new_xsd__base64Binary(aws.soap);
FILE *fd = fopen((argc > 3 ? argv[3] : "filename"), "rb");
if (fd == NULL) return 0;
struct stat sb;
if (!fstat(fileno(fd), &sb) && sb.st_size > 0) {
// We can get the length of the file, so we can stream it
// Set soap callbacks
aws.soap->fdimereadopen = dime_read_open;
aws.soap->fdimereadclose = dime_read_close;
aws.soap->fdimeread = dime_read;
//__ptr must be non-NULL. This is the handle in the callbacks
data->__ptr = (unsigned char*)fd;
data->__size = sb.st_size;
}
else {
// We don't know the size, so buffer it
int i;
data->__ptr = (unsigned char*)soap_malloc(aws.soap, MAX_FILE_SIZE);
for (i = 0; i < MAX_FILE_SIZE; i++) {
int c;
if ((c = fgetc(fd)) == EOF)
break;
data->__ptr[i] = c;
}
fclose(fd);
data->__size = i;
}
// Set DIME attachment with options
char type[] = "text/plain";
data->type = type;
soap_set_dime(aws.soap);
if (soap_set_dime_attachment(aws.soap, (char*)data->__ptr, data->__size, data->type, data->id, 0, data->options)) {
aws.soap_stream_fault(std::cerr);
return 0;
}
putObjectReq.ContentLength = data->__size;
// Store the result of the service
_s3__PutObjectResponse putObjectRes;
// Put object
if (aws.PutObject(&putObjectReq, putObjectRes)) {
aws.soap_stream_fault(std::cerr);
}
else if (putObjectRes.PutObjectResponse) {
s3__PutObjectResult &result = *putObjectRes.PutObjectResponse;
std::cout << "Object with key '" << putObjectReq.Key
<< "' put into bucket '" << putObjectReq.Bucket << "'" << std::endl;
std::cout << "\tEtag: " << result.ETag << std::endl;
std::cout << "\tLast modified: " << soap_dateTime2s(aws.soap, result.LastModified) << std::endl;
}
// Delete all managed data
aws.destroy();
return 0;
}
Compiling the PutObject(streaming) Example {#compiling2}
------------------------------------------
[command]
c++ -DWITH_OPENSSL -o putobjectstreaming putobjectstreaming.cpp \
soapAmazonS3SoapBindingProxy.cpp soapC.cpp stdsoap2.cpp -lssl -lcrypto
See the section titled [Compiling the CreateBucket Example](#compiling) for an explanation of the command.
In the final *.cpp* file, note that we check the command line arguments (`argv`) when setting `bucketName`, `keyName`, the filename, and `credentialsFile`. This allows the program to be called with arguments:
[command]
./putobjectstreaming BucketName KeyName filename path_to_credentials_file
Other AWS S3 API Functions With Examples {#other}
----------------------------------------
Calling different API functions will simply involve using different classes and setting the appropriate parameters.
The full SOAP API documentation for AWS S3 can be found [here](http://docs.aws.amazon.com/AmazonS3/latest/API/APISoap.html). It lists the operations available. You can also check the generated *aws-s3.h* file or the auto-generated [report](examples/aws/soapReadme.html) on it to get a list of the available gSOAP classes and functions (which are auto-generated from Amazon's WSDL/XSD).
I've written working example programs for the usage of the following AWS S3 API functions:
- `CreateBucket` (*createbucket.cpp*) (shown in this article)
- `ListBucket` (*listbucket.cpp*)
- `DeleteBucket` (*deletebucket.cpp*)
- `GetBucketAccessControlPolicy` (*setacp.cpp*)
- `SetBucketAccessControlPolicy` (*setacp.cpp*)
- `PutObjectInline` (*putobjectinline.cpp*)
- `PutObject `(non-streaming: *putobject.cpp*, streaming: *putobjectstreaming.cpp*) (shown in this article)
- `CopyObject` (*copyobject.cpp*)
- `GetObject` (*getobject.cpp*)
- `DeleteObject` (*deleteobject.cpp*)
These are all included in the *aws-s3-gsoap-examples.zip* file available on [Code Project](http://www.codeproject.com/Articles/1108296/How-to-Use-Amazon-Simple-Storage-Service-S-in-Cplu).
This should be more than enough to create a functional client to interact with AWS S3, and to figure out how to use the other auto-generated classes from gSOAP to call the other API operations. It also would be possible to write a wrapper class to make interacting with AWS S3 even easier.
[ To top](#)
Other Security Considerations {#other-sec}
-----------------------------
For security reasons, the default limit on DIME attachment size received is 8 MB. To increase the limit, compile with `-DSOAP_MAXDIMESIZE=nnn`, where `nnn` is the size in bytes. For example:
[command]
c++ -DSOAP_MAXDIMESIZE=104857600 -DWITH_OPENSSL -o yourapp yourapp.cpp \
soapAmazonS3SoapBindingProxy.cpp soapC.cpp stdsoap2.cpp -lssl -lcrypto
This sets the maximum size of DIME attachments that can be received to 100 MB.
[ To top](#)
]]>