AWS Tools for PowerShell is now generally available with version 4.0

[matteo-prosperi]

Dear AWS PowerShell Customers,

We are excited to announce the GA release of version 4 of our PowerShell modules!

AWS Tools for PowerShell is available in three different variants:

• AWS.Tools is the new modular variant that allows for faster import times and a smaller footprint. AWS.Tools is compatible with both PowerShell Core 6+ and Windows PowerShell 5.1 when .NET Framework 4.7.2 is installed. Starting with version 4.0 of AWS Tools for PowerShell, we suggest all users use AWS.Tools as their preferred option.
• AWSPowerShell.NetCore is the monolithic variant that supports all AWS services in a single large module. AWSPowerShell.NetCore is also compatible with both PowerShell Core 6+ and Windows PowerShell 3+ when .NET Framework 4.7.2 is installed.
• AWSPowerShell is the legacy variant for older systems which are either running Windows PowerShell 2 or cannot be updated to .NET Framework 4.7.2. Following the end of support of Windows Server 2008 R2, in January 2020, we plan to stop releasing new AWSPowerShell versions compatible with Windows PowerShell 2.

Version 4 of AWS Tools for PowerShell is now open sourced and available under the Apache version 2 license at our GitHub repo. Its highly modular architecture with a separate PowerShell module for each service provides numerous benefits, such as reduced download time, import time and memory usage.

Version 4.0 is highly backwards compatible with version 3.3 of AWS Tools for PowerShell and, in most scenarios, you can expect a seamless upgrade. As a good practice we always recommend thorough testing before upgrading production environments.

New modular variant of AWS Tools for PowerShell

AWS.Tools is the new modular variant of AWS Tools for PowerShell, which is preferred for most use cases. It has the same capabilities as AWSPowerShell and AWSPowerShell.NetCore and is compatible with all modern platforms:

• Windows PowerShell 5.1 when .NET Framework 4.7.2 (or newer) is installed
• PowerShell Core 6.0-6.2 on Windows, Linux and macOS.

A detailed description of all changes can be found in our preview announcement of AWS.Tools.

Faster import times, lower memory usage and more consistent auto-import

AWS.Tools has a modular architecture with a separate PowerShell module for each service. You can reduce download time, import time and memory usage by only installing modules your application requires, e.g. AWS.Tools.S3. Cmdlets will auto-import AWS.Tools modules, however AWSPowerShell.NetCore and AWSPowerShell might still require a manual Import-Module call due to the very large number of exported cmdlets.

If you want to use .NET objects before calling the first cmdlet of an AWS service, you still have to import the AWS.Tools module to let PowerShell know the associated .NET types:

PS> $filter = [Amazon.EC2.Model.Filter]@{Name="vpc-id";Values="vpc-1234abcd"}
  InvalidOperation: Unable to find type [Amazon.EC2.Model.Filter].
PS> Import-Module AWS.Tools.EC2
PS> $filter = [Amazon.EC2.Model.Filter]@{Name="vpc-id";Values="vpc-1234abcd"}
PS> Get-EC2Instance -Filter $filter -Select Reservations.Instances.InstanceId
  i-0123456789abcdefg
  i-0123456789hijklmn

Installation experience

In AWS.Tools cmdlets for each AWS service are provided by a separate module:

• AWS.Tools.EC2
• AWS.Tools.S3
• AWS.Tools.DirectoryService
• etc.

New versions of all the modules are released together with the same version number. Different versions of the AWS.Tools modules cannot be imported at the same time. Different variants of AWS Tools for PowerShell (AWS.Tools, AWSPowerShell.NetCore and AWSPowerShell) are not compatible with each other and cannot be imported at the same time.
To simplify the installation, update and removal of multiple AWS.Tools modules, we have created the AWS.Tools.Installer module. This example demonstrates how to use a single command to install multiple modules, sync existing modules to the current version and remove older versions.

PS> Install-AWSToolsModule AWS.Tools.EC2,AWS.Tools.S3 -CleanUp
  Confirm
  Are you sure you want to perform this action?
  Performing the operation "Install-AWSToolsModule" on target "AWS Tools version 4.0.0.0".
  [Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"):

  Installing module AWS.Tools.Common version 4.0.0.0
  Installing module AWS.Tools.EC2 version 4.0.0.0
  Installing module AWS.Tools.Glacier version 4.0.0.0
  Installing module AWS.Tools.S3 version 4.0.0.0

  Uninstalling AWS.Tools version 3.3.618.0
  Uninstalling module AWS.Tools.Glacier
  Uninstalling module AWS.Tools.S3
  Uninstalling module AWS.Tools.SimpleNotificationService
  Uninstalling module AWS.Tools.SQS
  Uninstalling module AWS.Tools.Common

Using AWS.Tools.Installer is optional, AWS.Tools modules can also be installed and uninstalled directly from PowerShell Gallery.

Mandatory parameters

The new AWS.Tools modules now declare and enforce mandatory cmdlet parameters. When an AWS Service declares that an API parameter is required, PowerShell will prompt you for the missing corresponding cmdlet parameter. For backward compatibility reasons we did not extend this feature to the AWSPowerShell.NetCore or AWSPowerShell modules.

PS> Get-EC2InstanceAttribute -Attribute instanceType -Select InstanceAttribute.InstanceType
  cmdlet Get-EC2InstanceAttribute at command pipeline position 1
  Supply values for the following parameters:
  InstanceId: i-0123456789abcdefg
  
  t2.micro

You now have an option to pass $null as a value for all mandatory parameters, including value type parameters, such as numbers and dates. This is a minor change meant to allow bypassing the prompt in case we mistakenly marked a parameter as mandatory. This change is transparent for users, and cmdlets behavior remains unchanged.

This example demonstrates how $null is explicitly passed to two mandatory parameters. As you can see the cmdlet does not prompt for a value, and, as a consequence, the request fails service-side validation.

PS> Get-EC2InstanceAttribute -InstanceId $null -Attribute $null
  WARNING: You are passing $null as a value for parameter Attribute which is marked as required. In case you believe this parameter was incorrectly marked as required, report this by opening an issue at https://github.com/aws/aws-tools-for-powershell/issues.
  WARNING: You are passing $null as a value for parameter InstanceId which is marked as required. In case you believe this parameter was incorrectly marked as required, report this by opening an issue at https://github.com/aws/aws-tools-for-powershell/issues.
  Get-EC2InstanceAttribute : The request must contain the parameter instanceId

Simplified and extended auto-iteration

AWS Tools for PowerShell support auto-pagination. Cmdlets like Get-S3Object will internally use, if necessary, multiple service calls in order to retrieve all the values.
In AWSPowerShell.NetCore and AWSPowerShell the -MaxItems parameter allows to limit the number of items returned by some cmdlets. This behavior is now considered obsolete and is not available in AWS.Tools. You can instead pipe the output of the cmdlet into | select -first $n.

PS> Get-S3Object -BucketName mybucket -Select S3Objects.Key | select -first 1
  file1.txt

Leveraging the new -Select parameter and this simplified pagination approach, we were able to enable auto-pagination in AWS.Tools for an additional 70 cmdlets that require manual pagination in AWSPowerShell.NetCore and AWSPowerShell.

Additional information about the reasons for the auto-iteration changes and details about the new features in version 4 of AWS Tools for PowerShell can be found in our pre-announcement.

New cmdlet Get-AWSService

To simplify discovery of the available modules, we have added the new Get-AWSService cmdlet.

PS> Get-AWSService
  Service : ACMPCA
  CmdletNounPrefix : PCA
  ModuleName : AWS.Tools.ACMPCA
  SDKAssemblyVersion : 3.3.101.56
  ServiceName : AWS Certificate Manager Private Certificate Authority

  Service : AlexaForBusiness
  CmdletNounPrefix : ALXB
  ModuleName : AWS.Tools.AlexaForBusiness
  SDKAssemblyVersion : 3.3.106.26
  ServiceName : Alexa For Business
  ...

AWS Lambda functions

If you are using AWS Tools for PowerShell in AWS Lambda, you can now switch to the new AWS.Tools modules. They will help lower cold start time and memory usage.

You will need to change the #Requires statement from AWSPowerShell.NetCore to the AWS.Tools module you wish to use (e.g. AWS.Tools.S3). You will also need to add an extra #Requires statement to use AWS.Tools.Common.
Change the line:

#Requires -Modules @{ModuleName='AWSPowerShell.NetCore';ModuleVersion='4.0.0.0'}

To:

#Requires -Modules @{ModuleName='AWS.Tools.Common';ModuleVersion='4.0.0.0'}
#Requires -Modules @{ModuleName='AWS.Tools.S3';ModuleVersion='4.0.0.0'}

Version 4 features of AWS Tools for Powershell

The following new features are available in all 3 variants: AWS.Tools, AWSPowerShell.NetCore and AWSPowerShell.

Select parameter

In version 4.0 most cmdlets have a new parameter: -Select. Select can be used to change the value returned by the cmdlet.
For example the service API used by Get-S3Object returns a ListObjectsResponse object but the cmdlet is configured to return only the S3Objects field, resulting in:

PS> Get-S3Object -BucketName mybucket
  ETag : "01234567890123456789012345678901234"
  BucketName : mybucket
  Key : file1.txt
  LastModified : 9/30/2019 1:31:40 PM
  Owner : Amazon.S3.Model.Owner
  Size : 568
  StorageClass : STANDARD

  ETag : "01234567890123456789012345678901235"
  BucketName : mybucket
  Key : file2.txt
  LastModified : 7/15/2019 9:36:54 AM
  Owner : Amazon.S3.Model.Owner
  Size : 392
  StorageClass : STANDARD

Sometimes the service response contains additional data and metadata that might be of interest. Now you can specify -Select * to receive the full API response.

PS> Get-S3Object -BucketName mybucket -Select *
  IsTruncated    : False
  NextMarker     :
  S3Objects      : {file1.txt, file2.txt}
  Name           : mybucket
  Prefix         :
  MaxKeys        : 1000
  CommonPrefixes : {}
  Delimiter      :

You can also specify the path to a nested result property like -Select S3Objects.Key.

PS> Get-S3Object -BucketName mybucket -Select S3Objects.Key
  file1.txt
  file2.txt

In certain situations it may be useful to return a cmdlet parameter. This can be achieved with -Select ^ParameterName. This feature supplants the -PassThru parameter which is still available but deprecated.

PS> Get-S3Object -BucketName mybucket -Select S3Objects.Key |
>> Write-S3ObjectTagSet -Select ^Key -BucketName mybucket -Tagging_TagSet @{ Key='key'; Value='value'}
  file1.txt
  file2.txt

Easy to use Stream parameters

Parameters of type Stream or byte[] can now accept string, string[] or FileInfo values.
For example, you can use any of the following:

PS> Invoke-LMFunction -FunctionName MyTestFunction -PayloadStream '{
>>   "some": "json"
>> }'

PS> Invoke-LMFunction -FunctionName MyTestFunction -PayloadStream (ls .\file.json)


PS> Invoke-LMFunction -FunctionName MyTestFunction -PayloadStream @('{', '"some": "json"', '}')

All strings are converted to byte[] using UTF8 encoding.

Extending Pipe by property name

In order to provide a consistent user experience pipeline input by Property Name is now enabled for all parameters.

PS> [pscustomobject] @{ BucketName='myBucket'; Key='file1.txt'; PartNumber=1 } | Get-S3ObjectMetadata

Static common parameters

In previous versions of AWS Tools for PowerShell, common parameters (AccessKey, SecretKey, ProfileName, Region, etc.) used to be dynamic while all other parameters were static. This could create problems because PowerShell binds static parameters before dynamic ones.
For example, when calling Get-EC2Region -Region us-west-2, PowerShell used to bind us-west-2 to the -RegionName static parameter instead of the -Region dynamic parameter.
In order to improve consistency in version 4.0 of AWS Tools for PowerShell all parameters are static. This may have an effect on the behavior of scripts if they are using:

• AccessKey parameter of Remove-IAMAccessKey, Get-IAMAccessKeyLastUsed and Update-IAMAccessKey
• Credential parameter of New-AG2Integration, Update-AG2Integration, New-IOTRoleAlias and Update-IOTRoleAlias
• Region parameter of New-AGDomainName, Get-DDBGlobalTableList, Get-EC2Region, Get-RDSSourceRegion and Update-R53HealthCheck

Making the project open-source

With the release of version 4 of AWS Tools for PowerShell, we are also making the source code available under the Apache version 2 license at our GitHub repo.
You can build the source code by running msbuild .\buildtools\build.proj on a Windows machine. You will also need reference assemblies for .NET Framework 3.5 and the .NET Core SDK version 2.1.

Removing deprecated features

The following, previously deprecated, features are removed in version 4:

• Removed the -Terminate parameter from the Stop-EC2Instance cmdlet. Use Remove-EC2Instance instead.
• Removed the -ProfileName parameter from the Clear-AWSCredential cmdlet. Use Remove-AWSCredentialProfile instead.
• Removed cmdlets Import-EC2Instance and Import-EC2Volume.

End of support for Windows PowerShell 2

Following the end of support of Windows Server 2008 R2, in January 2020, we plan to stop releasing new AWSPowerShell versions compatible with Windows PowerShell 2. We will also make .NET Framework 4.5 a pre-requisite for running new releases of AWSPowerShell.
The existing versions of AWSPowerShell which are compatible with Windows PowerShell 2 and .NET Framework 3.5 will continue to be available for download from PowerShell Gallery.

Removed type extensions

AWS.Tools doesn’t include undocumented type extensions which are available in AWSPowerShell and AWSPowerShell.NetCore. These type extensions provide aliases for fields of .NET objects, mostly related to the EC2 service.

Feedback

We look forward to receiving your feedback about the new major version of AWS Tools for PowerShell and your ideas for new features. You can contact us by opening an issue on the aws-tools-for-powershell GitHub repo.