[Backport 2.x] Fixing GetSourceRequest (#403)

* Cleaning up changelog and updating documentation (#401) * Cleaning up changelog Signed-off-by: Vacha Shah <vachshah@amazon.com> * Updating compatibility matrix Signed-off-by: Vacha Shah <vachshah@amazon.com> --------- Signed-off-by: Vacha Shah <vachshah@amazon.com> * Fixing GetSourceRequest (#402) * Fixing GetSourceRequest source field Signed-off-by: Vacha Shah <vachshah@amazon.com> * Adding sample code for get source api Signed-off-by: Vacha Shah <vachshah@amazon.com> * Adding changelog Signed-off-by: Vacha Shah <vachshah@amazon.com> --------- Signed-off-by: Vacha Shah <vachshah@amazon.com> --------- Signed-off-by: Vacha Shah <vachshah@amazon.com>
opensearch-project · Oct 23, 2023 · 8564a2f · 8564a2f
1 parent 8f11c2d
commit 8564a2f
Show file tree

Hide file tree

Showing 6 changed files with 117 additions and 47 deletions.
diff --git a/.github/workflows/test-compatibility.yml b/.github/workflows/test-compatibility.yml
@@ -24,6 +24,7 @@ jobs:
           - { opensearch_version: 2.8.0 }
           - { opensearch_version: 2.9.0 }
           - { opensearch_version: 2.10.0 }
+          - { opensearch_version: 2.11.0 }
     steps:
       - uses: actions/checkout@v3
         with: { fetch-depth: 1 }

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,61 +6,38 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### Dependencies
 
-- Bumps `github.com/aws/aws-sdk-go` from 1.44.180 to 1.45.26
-- Bumps `github.com/aws/aws-sdk-go-v2` from 1.17.4 to 1.21.1
-- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.19.0
-- Bumps `github.com/stretchr/testify` from 1.8.1 to 1.8.4
+- Bumps `github.com/aws/aws-sdk-go` from 1.44.263 to 1.45.26
+- Bumps `github.com/aws/aws-sdk-go-v2` from 1.18.0 to 1.21.1
+- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.25 to 1.19.0
+- Bumps `github.com/stretchr/testify` from 1.8.2 to 1.8.4
 - Bumps `golang.org/x/net` from 0.7.0 to 0.17.0
 - Bumps `github.com/golangci/golangci-lint-action` from 1.53.3 to 1.54.2
 
 ### Added
 
-- Adds implementation of Data Streams API ([#257](https://github.com/opensearch-project/opensearch-go/pull/257))
-- Adds Point In Time API ([#253](https://github.com/opensearch-project/opensearch-go/pull/253))
-- Adds InfoResp type ([#253](https://github.com/opensearch-project/opensearch-go/pull/253))
-- Adds markdown linter ([#261](https://github.com/opensearch-project/opensearch-go/pull/261))
-- Adds testcases to check upsert functionality ([#269](https://github.com/opensearch-project/opensearch-go/pull/269))
-- Adds @Jakob3xD to co-maintainers ([#270](https://github.com/opensearch-project/opensearch-go/pull/270))
-- Adds dynamic type to \_source field ([#285](https://github.com/opensearch-project/opensearch-go/pull/285))
-- Adds testcases for Document API ([#285](https://github.com/opensearch-project/opensearch-go/pull/285))
-- Adds `index_lifecycle` guide ([#287](https://github.com/opensearch-project/opensearch-go/pull/287))
-- Adds `bulk` guide ([#292](https://github.com/opensearch-project/opensearch-go/pull/292))
-- Adds `search` guide ([#291](https://github.com/opensearch-project/opensearch-go/pull/291))
-- Adds `document_lifecycle` guide ([#290](https://github.com/opensearch-project/opensearch-go/pull/290))
-- Adds `index_template` guide ([#289](https://github.com/opensearch-project/opensearch-go/pull/289))
-- Adds `advanced_index_actions` guide ([#288](https://github.com/opensearch-project/opensearch-go/pull/288))
-- Adds testcases to check UpdateByQuery functionality ([#304](https://github.com/opensearch-project/opensearch-go/pull/304))
-- Adds additional timeout after cluster start ([#303](https://github.com/opensearch-project/opensearch-go/pull/303))
-- Adds docker healthcheck to auto restart the container ([#315](https://github.com/opensearch-project/opensearch-go/pull/315))
+- Adds `Err()` function to Response for detailed errors ([#246](https://github.com/opensearch-project/opensearch-go/pull/246))
 - Adds golangci-lint as code analysis tool ([#313](https://github.com/opensearch-project/opensearch-go/pull/313))
 
 ### Changed
 
-- Uses `[]string` instead of `string` in `SnapshotDeleteRequest` ([#237](https://github.com/opensearch-project/opensearch-go/pull/237))
 - Removes the need for double error checking ([#246](https://github.com/opensearch-project/opensearch-go/pull/246))
-- Updates workflows to reduce CI time, consolidate OpenSearch versions, update compatibility matrix ([#242](https://github.com/opensearch-project/opensearch-go/pull/242))
-- Moved @svencowart to emeritus maintainers ([#270](https://github.com/opensearch-project/opensearch-go/pull/270))
-- Read, close and replace the http Reponse Body ([#300](https://github.com/opensearch-project/opensearch-go/pull/300))
 - Updated and adjusted golangci-lint, solve linting complains for signer ([#352](https://github.com/opensearch-project/opensearch-go/pull/352))
 - Solve linting complains for opensearchtransport ([#353](https://github.com/opensearch-project/opensearch-go/pull/353))
-- Updated Developer guide to include docker build instructions ([#385](https://github.com/opensearch-project/opensearch-go/pull/385)
+- Updated Developer guide to include docker build instructions ([#385](https://github.com/opensearch-project/opensearch-go/pull/385))
 - Test against version 2.9.0,2.10.0, run tests in all branches, change intergration tests to wait for OpenSearch to start ([#392](https://github.com/opensearch-project/opensearch-go/pull/392))
 - Makefile: use docker golangci-lint, run integration test on `.` folder, change coverage generation ([#392](https://github.com/opensearch-project/opensearch-go/pull/392)) 
 
 ### Deprecated
 
-- Deprecate legacy API /_template ([#151](https://github.com/opensearch-project/opensearch-go/issues/151))
+- Deprecate legacy API /_template ([#390](https://github.com/opensearch-project/opensearch-go/pull/390))
 
 ### Removed
 
-- Removes info call before performing every request ([#219](https://github.com/opensearch-project/opensearch-go/pull/219))
-
 ### Fixed
 
-- Corrects curl logging to emit the correct URL destination ([#101](https://github.com/opensearch-project/opensearch-go/pull/101))
-- Corrects handling of errors without an error response body ([#286](https://github.com/opensearch-project/opensearch-go/pull/286))
 - Corrects AWSv4 signature on DataStream Stats with no index name specified ([#338](https://github.com/opensearch-project/opensearch-go/pull/338))
+- Fixed GetSourceRequest `Source` field and deprecated the `Source` parameter ([#402](https://github.com/opensearch-project/opensearch-go/pull/402))
 
 ### Security
 
-[Unreleased]: https://github.com/opensearch-project/opensearch-go/compare/v2.2.0...HEAD
+[Unreleased]: https://github.com/opensearch-project/opensearch-go/compare/v2.3.0...HEAD
diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md
@@ -7,12 +7,8 @@ The below matrix shows the compatibility of the [`opensearch-go`](https://pkg.go
 
 | Client Version | OpenSearch Version |
 | -------------- | ------------------ |
-| 1.0.0          | 1.0.0-1.0.1        |
-| 1.1.0          | 1.1.0-1.3.1        |
-| 2.0.0          | 2.0.0-2.1.0        |
-| 2.0.1          | 2.0.0-2.1.0        |
-| 2.1.0          | 2.0.0-2.5.0        |
-| 2.2.0          | 2.0.0-2.5.0        |
+| 1.x.0          | 1.x                |
+| 2.x.0          | 1.3.13-2.11.0      |
 
 ## Upgrading
 

diff --git a/guides/search.md b/guides/search.md
@@ -186,6 +186,42 @@ log.Printf("deleted pits: [%+v]", delpits)
 
 Note that a point-in-time is associated with an index or a set of index. So, when performing a search with a point-in-time, you DO NOT specify the index in the search.
 
+## Source API
+
+The source API returns the source of the documents with included or excluded fields. The following example returns all fields from document source in the `movies` index:
+
+```go
+getSourceRequest := opensearchapi.GetSourceRequest{
+    Index:      "movies",
+    DocumentID: "1",
+}
+getSourceResponse, err := getSourceRequest.Do(context.Background(), client)
+if err != nil {
+    log.Printf("error occurred: [%s]", err.Error())
+}
+log.Printf("source: [%+v]", getSourceResponse)
+```
+
+To include certain fields in the source response, use `SourceIncludes` or `Source`(this field is deprecated and `SourceIncludes` is recommended to be used instead). To get only required fields:
+
+```go
+getSourceRequest = opensearchapi.GetSourceRequest{
+    Index:      "movies",
+    DocumentID: "1",
+    SourceIncludes:     []string{"title"},
+}
+```
+
+To exclude certain fields in the source response, use `SourceExcludes` as follows:
+
+```go
+getSourceRequest = opensearchapi.GetSourceRequest{
+    Index:      "movies",
+    DocumentID: "1",
+    SourceExcludes: []string{"title"},
+}
+```
+
 ## Cleanup
 
 ```go

diff --git a/opensearchapi/api.get_source.go b/opensearchapi/api.get_source.go
@@ -60,7 +60,8 @@ type GetSourceRequest struct {
 	Realtime       *bool
 	Refresh        *bool
 	Routing        string
-	Source         interface{}
+	// Deprecated: This parameter is similar to SourceIncludes, please use that instead.
+	Source         []string
 	SourceExcludes []string
 	SourceIncludes []string
 	Version        *int
@@ -113,12 +114,8 @@ func (r GetSourceRequest) Do(ctx context.Context, transport Transport) (*Respons
 		params["routing"] = r.Routing
 	}
 
-	if source, ok := r.Source.(bool); ok {
-		params["_source"] = strconv.FormatBool(source)
-	} else if source, ok := r.Source.(string); ok && source != "" {
-		params["_source"] = source
-	} else if sources, ok := r.Source.([]string); ok && len(sources) > 0 {
-		params["_source"] = strings.Join(sources, ",")
+	if len(r.Source) > 0 {
+		params["_source"] = strings.Join(r.Source, ",")
 	}
 
 	if len(r.SourceExcludes) > 0 {
@@ -236,9 +233,9 @@ func (f GetSource) WithRouting(v string) func(*GetSourceRequest) {
 	}
 }
 
-// WithSource - true or false to return the _source field or not, or a list of fields to return.
+// WithSource - a list of fields to return. This field is deprecated, please use SourceIncludes instead.
 //
-func (f GetSource) WithSource(v interface{}) func(*GetSourceRequest) {
+func (f GetSource) WithSource(v ...string) func(*GetSourceRequest) {
 	return func(r *GetSourceRequest) {
 		r.Source = v
 	}

diff --git a/opensearchapi/opensearchapi_integration_test.go b/opensearchapi/opensearchapi_integration_test.go
@@ -36,6 +36,7 @@ import (
 	"strings"
 	"testing"
 	"time"
+	"github.com/stretchr/testify/assert"
 
 	"github.com/opensearch-project/opensearch-go/v2"
 	"github.com/opensearch-project/opensearch-go/v2/opensearchapi"
@@ -129,6 +130,68 @@ func TestAPI(t *testing.T) {
 			t.Fatalf("Failed to index data: %s", err)
 		}
 
+		// Add a document to the index.
+		document := strings.NewReader(`{
+			"title": "Moneyball",
+			"director": "Bennett Miller",
+			"year": "2011"
+		}`)
+
+		docId := "1"
+		req := opensearchapi.IndexRequest{
+			Index:      "test",
+			DocumentID: docId,
+			Body:       document,
+		}
+		insertResponse, err := req.Do(context.Background(), client)
+		if err != nil {
+			fmt.Println("failed to insert document ", err)
+		}
+		defer insertResponse.Body.Close()
+
+		// Get the document with source filters
+		getSourceRequest := opensearchapi.GetSourceRequest{
+			Index:      "test",
+			DocumentID: "1",
+			Source:     []string{"title"},
+		}
+		getSourceResponse, err := getSourceRequest.Do(context.Background(), client)
+		if err != nil {
+			fmt.Println("failed to get source ", err)
+		}
+		assert.Equal(t, 200, getSourceResponse.StatusCode)
+		assert.Contains(t, getSourceResponse.String(), "title")
+		assert.NotContains(t, getSourceResponse.String(), "director")
+		defer getSourceResponse.Body.Close()
+
+		getSourceRequest = opensearchapi.GetSourceRequest{
+			Index:      "test",
+			DocumentID: "1",
+			SourceIncludes:     []string{"title"},
+		}
+		getSourceResponse, err = getSourceRequest.Do(context.Background(), client)
+		if err != nil {
+			fmt.Println("failed to get source ", err)
+		}
+		assert.Equal(t, 200, getSourceResponse.StatusCode)
+		assert.Contains(t, getSourceResponse.String(), "title")
+		assert.NotContains(t, getSourceResponse.String(), "director")
+		defer getSourceResponse.Body.Close()
+
+		getSourceRequest = opensearchapi.GetSourceRequest{
+			Index:      "test",
+			DocumentID: "1",
+			SourceExcludes: []string{"title"},
+		}
+		getSourceResponse, err = getSourceRequest.Do(context.Background(), client)
+		if err != nil {
+			fmt.Println("failed to get source ", err)
+		}
+		assert.Equal(t, 200, getSourceResponse.StatusCode)
+		assert.Contains(t, getSourceResponse.String(), "director")
+		assert.NotContains(t, getSourceResponse.String(), "title")
+		defer getSourceResponse.Body.Close()
+
 		// Launch reindexing task with wait_for_completion=false
 		//
 		res, err = client.Reindex(