Preview of version 4 features of AWS Tools for PowerShell

[matteo-prosperi]

In August, the AWS .NET Team released the first preview of AWS.Tools: the modular version of AWS Tools for PowerShell. As we are close to declaring AWS.Tools ready for production use, we can now announce that the generally available version of AWS.Tools will be part of a major version update of AWS Tools for PowerShell (version 4.0).

We have included a preview of the new v4 features in today’s AWS.Tools prerelease (version 3.3.618). So, if you are already testing the new modular version of AWS Tools for PowerShell, you can try out the new features as well. When v4 is released, most new features will be available to users of the AWSPowerShell and AWSPowerShell.NetCore modules as well.

AWS Tools for PowerShell version 4.0 will be a highly backwards compatible update to AWS Tools for PowerShell 3.3. This new major version includes some significant improvements while maintaining existing cmdlet behavior. Your scripts should still work when upgrading to the new version but we recommend testing them thoroughly before upgrading production environments.

AWS Tools for PowerShell is available in three different variants:

• AWS.Tools is the new modular variant which allow faster import times and a smaller footprint. AWS.Tools is compatible with both PowerShell Core 6+ and Windows PowerShell 5.1 when .NET Framework 4.7.2 is installed. Starting with version 4.0 of AWS Tools for PowerShell, we suggest all users use AWS.Tools as their preferred option.
• AWSPowerShell.NetCore is the monolithic variant that supports all AWS services in a single large module. AWSPowerShell.NetCore is also compatible with both PowerShell Core 6+ and Windows PowerShell 3+ when .NET Framework 4.7.2 is installed.
• AWSPowerShell is a legacy variant for older systems which are either running Windows PowerShell 2 or cannot be updated to .NET Framework 4.7.2. Following the end of support of Windows Server 2008 R2, in 2020, we plan to stop releasing new AWSPowerShell versions compatible with Windows PowerShell 2.

The following are the most significant changes in version 4.0 of AWS Tools for PowerShell.

Select Parameter

Most cmdlets in version 4.0 have a new parameter: -Select. Select can be used to change the value returned by the cmdlet.
For example the service API used by Get-S3Object returns a ListObjectsResponse object but the cmdlet is configured to return only the S3Objects field. Resulting in:

PS> Get-S3Object -BucketName mybucket

ETag : "01234567890123456789012345678901234"
BucketName : mybucket
Key : file1.txt
LastModified : 9/30/2019 1:31:40 PM
Owner : Amazon.S3.Model.Owner
Size : 568
StorageClass : STANDARD

ETag : "01234567890123456789012345678901235"
BucketName : mybucket
Key : file2.txt
LastModified : 7/15/2019 9:36:54 AM
Owner : Amazon.S3.Model.Owner
Size : 392
StorageClass : STANDARD

Sometimes the service response contains additional data and metadata that might be of interest. Now you can specify -Select * to receive the full API response.

PS> Get-S3Object -BucketName mybucket -Select *

IsTruncated    : False
NextMarker     :
S3Objects      : {file1.txt, file2.txt}
Name           : mybucket
Prefix         :
MaxKeys        : 1000
CommonPrefixes : {}
Delimiter      :

You can also specify the path to a nested result property like -Select S3Objects.Key.

PS> Get-S3Object -BucketName mybucket -Select S3Objects.Key
file1.txt
file2.txt

In certain situations it may be useful to return a cmdlet parameter. This can be achieved with -Select ^ParameterName. This feature supplants the -PassThru parameter which is still available but deprecated.

PS> Get-S3Object -BucketName mybucket -Select S3Objects.Key |
>> Write-S3ObjectTagSet -Select ^Key -BucketName mybucket -Tagging_TagSet @{ Key='key'; Value='value'}
file1.txt
file2.txt

Simplified auto-iteration

AWS Tools for PowerShell support auto-pagination. Cmdlets like Get-S3Object will internally use multiple service calls, if necessary, in order to retrieve all the values.
The AWSPowerShell.NetCore and AWSPowerShell modules have special behavior that allows use of the -MaxItems parameter to limit the number of returned items. This behavior is now considered obsolete and is not available in AWS.Tools. You can instead pipe the output of the cmdlet into | select -first $n.

PS> Get-S3Object -BucketName mybucket -MaxItems 1 -Select S3Objects.Key

WARNING: AWSPowerShell and AWSPowerShell.NetCore use the MaxKey parameter to limit the total number of items returned by the cmdlet. This behavior is obsolete and will be removed in a future version of these modules. Pipe the output of this cmdlet into Select-Object -First to terminate retrieving data pages early and control the number of items returned. AWS.Tools already implements the new behavior of simply passing MaxKey to the service to specify how many
items should be returned by each service call.

file1.txt

PS> Get-S3Object -BucketName mybucket -Select S3Objects.Key | select -first 1
file1.txt

There are multiple reasons for this change:

• This functionality is not guaranteed directly by the services, so it is not always possible to return the exact number of items requested.
• Not all services allow small values for MaxItems, so requesting just 1 or 2 items frequently results in an error.
• Not all paginated service operations have a MaxItems parameter.
• The usage of -MaxItems and -Select together can be confusing.

Leveraging the new -Select parameter and this simplified pagination approach, we were able to enable auto-pagination for an additional 70 cmdlets in AWS.Tools.

Auto-paginated cmdlets also have a new -NoAutoPagination parameter which provides an explicit way to disable auto-pagination.

Easier to use Stream parameters

Parameters of type Stream or byte[] can now accept string, string[] or FileInfo values.
For example, you can use any of the following:

PS> Invoke-LMFunction -FunctionName MyTestFunction -PayloadStream '{
>>   "some": "json"
>> }'

PS> Invoke-LMFunction -FunctionName MyTestFunction -PayloadStream (ls .\file.json)


PS> Invoke-LMFunction -FunctionName MyTestFunction -PayloadStream @('{', '"some": "json"', '}')

All strings are converted to byte[] using UTF8 encoding.

Extending Pipe by property name

In order to make the user experience more consistent, pipeline input by Property Name is now enabled for all parameters.

PS> [pscustomobject] @{ BucketName='myBucket'; Key='file1.txt'; PartNumber=1 } | Get-S3ObjectMetadata

Static common parameters

In previous versions of AWS Tools for PowerShell, common parameters (AccessKey, SecretKey, ProfileName, Region, etc.) used to be dynamic while all other parameters were static. This could create problems because PowerShell binds static parameters before dynamic ones.

For example, when calling Get-EC2Region -Region us-west-2, PowerShell used to bind us-west-2 the -RegionName static parameter instead of the -Region dynamic parameter. Obviously this is confusing for users.

In order to improve consistency in version 4.0 of AWS Tools for PowerShell, all parameters are static.

Making all parameters nullable

Value type parameters (numbers and dates) now accept $null. This change should be transparent for users and all cmdlets are expected to behave exactly like before.
This is a minor change meant to allow bypassing the prompt for mandatory parameters in case we mistakenly marked a parameter as mandatory. Mandatory parameters are enforced in AWS.Tools only.

The following will now attempt to call the EC2 service without parameters, bypassing the client-side validation (and will fail server-side).

PS> Get-EC2InstanceAttribute -InstanceId $null -Attribute $null

WARNING: You are passing $null as a value for parameter Attribute which is marked as required.
In case you believe this parameter was incorrectly marked as required, report this by opening
an issue at https://github.com/aws/aws-tools-for-powershell/issues.

WARNING: You are passing $null as a value for parameter InstanceId which is marked as required.
In case you believe this parameter was incorrectly marked as required, report this by opening
an issue at https://github.com/aws/aws-tools-for-powershell/issues.

Get-EC2InstanceAttribute : The request must contain the parameter instanceId

Removing deprecated features

The following, previously deprecated, features will be removed:

• Removed the -Terminate parameter from the Stop-EC2Instance cmdlet. Use Remove-EC2Instance instead.
• Removed the -ProfileName parameter from the Clear-AWSCredential cmdlet. Use Remove-AWSCredentialProfile instead.
• Removed cmdlets Import-EC2Instance and Import-EC2Volume.

Conclusion

You can find more information about AWS.Tools in our previous GitHub announcement.

As always all modules are available on PowerShell Gallery. If you experience any issue with the new preview modules or you have any feedback, let us know by creating an issue in this repository.