Saving an Undefined value to a ParseObject Field that is undefined already, creates a null pointer field in MongoDb Atlas Document which creates query issues

[432player]

New Issue Checklist

• [ x] I am not disclosing a vulnerability.
• [ x] I am not just asking a question.
• [ x] I have searched through existing issues.
• [ x] I can reproduce the issue with the latest version of Parse Server.

Issue Description

Saving an undefined value into an existing ParseObject field, creates a null pointer in the mongoDb Atlas document.
For Example:
I have a ParseObject "Form" with a field "student" which is a _User object.
When I set "undefined" into the student field and then save the Form object it creates a null pointer field which is not detectable through the parse-server but appears as null in mongoDB as such -> _p_student null

The moment this pointer is created with null value, it is impossible to use doesNotExist and exists in a query because it believes the object exists. While in the parse dashboard it appears as undefined, but acts as an existing object when working with exists & doesNotExist in a query.

Steps to reproduce

Easy, just save an undefined value into an object field of an existing ParseObject and save the object. It will seem as if the value is undefined, but don't be fooled. the dashboard even will show you that it's undefined and when using .get('student') it witll be undefined in code. BUT, when you want to do "exists" in a query it will assume it exists, because when going to mongodb atlas you will find a _p_field_name : null and thus disrupting the exists/doesNotExists functionality in a query.

Actual Outcome

var formQuery = new Parse.Query('Form');
formQuery.doesNotExist('student');
formQuery.find({ useMasterKey: true })
.then(function (formObjects) {
//The object where I've inserted an undefined value into student will not return in this query
})

Expected Outcome

var formQuery = new Parse.Query('Form');
formQuery.doesNotExist('student');
formQuery.find({ useMasterKey: true })
.then(function (formObjects) {
//Having the object which I saved undefined into student should still appear here
})

Environment

NodeJs

Server

• Parse Server version: 6.2.0
• Operating system: NodeJs
• Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Heroku

Database

• System (MongoDB or Postgres): MongoDB
• Database version: 6.0
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): MongoDB Atlas

Client

• SDK (iOS, Android, JavaScript, PHP, Unity, etc): JavaScript
• SDK version: 6.2.0
  "parse": "4.0.1",
  "parse-server": "^6.2.0",

[parse-github-assistant]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[cbaker6]

This is already supported on the Parse-Server and is not a bug. For example, issuing the following queries via REST work fine:

// yolo == null
"{\"_method\":\"GET\",\"limit\":100,\"skip\":0,\"where\":{\"yolo\":null}}"
// yolo == null, alternate way
"{\"_method\":\"GET\",\"limit\":100,\"skip\":0,\"where\":{\"yolo\":{\"$eq\":null}}}"
// yolo != null
"{\"_method\":\"GET\",\"limit\":100,\"skip\":0,\"where\":{\"yolo\":{\"$ne\":null}}}"

Actual tests already implemented on the server, below are some of them:

parse-server/spec/ParseQuery.spec.js

Lines 1293 to 1309 in 9674d4a

	it('where $eq null queries (rest)', done => {
	const options = Object.assign({}, masterKeyOptions, {
	qs: {
	where: JSON.stringify({ field: { $eq: null } }),
	},
	});
	const obj1 = new TestObject({ field: false });
	const obj2 = new TestObject({ field: null });
	Parse.Object.saveAll([obj1, obj2]).then(() => {
	return request(Object.assign({ url: Parse.serverURL + '/classes/TestObject' }, options)).then(
	resp => {
	equal(resp.data.results.length, 1);
	done();
	}
	);
	});
	});

parse-server/spec/ParseQuery.spec.js

Lines 108 to 126 in 9674d4a

	it('searching for not null', function (done) {
	const baz = new TestObject({ foo: null });
	const qux = new TestObject({ foo: 'qux' });
	const qux2 = new TestObject({});
	Parse.Object.saveAll([baz, qux, qux2]).then(function () {
	const query = new Parse.Query(TestObject);
	query.notEqualTo('foo', null);
	query.find().then(function (results) {
	equal(results.length, 1);
	qux.set('foo', null);
	qux.save().then(function () {
	query.find().then(function (results) {
	equal(results.length, 0);
	done();
	});
	});
	});
	});
	});

parse-server/spec/ParseQuery.spec.js

Lines 3824 to 3840 in 9674d4a

	// PG don't support creating a null column
	it_exclude_dbs(['postgres'])('querying for null value', done => {
	const obj = new Parse.Object('TestObject');
	obj.set('aNull', null);
	obj
	.save()
	.then(() => {
	const query = new Parse.Query('TestObject');
	query.equalTo('aNull', null);
	return query.find();
	})
	.then(results => {
	expect(results.length).toEqual(1);
	expect(results[0].get('aNull')).toEqual(null);
	done();
	});
	});

The actual issue reported is that the JS SDK (and other SDK's) doesn't support these type of query constraints directly and requires equalTo and notEqualTo null instead (note that you can still run into parse-community/Parse-SDK-JS#1372 on the JS SDK). The exists and doesNotExist query constraints work correctly, the JS SDK needs to add two additional constraints to support the REST commands I posted. Currently, the Swift SDK is the only Parse SDK that I know of to support the constraints (isNull and isNotNull).

Note that the differences between (isNull, isNotNull) and (doesNotExist, exists) is only relevant when using MongoDB. For Postgres, using either results in the same outcome respectively, see parse-community/Parse-Swift#308 for more info. Here's how the implementations look in the Swift SDK:

/**
 Add a constraint that requires that a key is equal to **null** or **undefined**.
 - parameter key: The key that the value is stored in.
 - returns: The same instance of `QueryConstraint` as the receiver.
 */
public func isNull (key: String) -> QueryConstraint {
    QueryConstraint(key: key, isNull: true)
}

/**
 Add a constraint that requires that a key is not equal to **null** or **undefined**.
 - parameter key: The key that the value is stored in.
 - returns: The same instance of `QueryConstraint` as the receiver.
 */
public func isNotNull (key: String) -> QueryConstraint {
    QueryConstraint(key: key, comparator: .notEqualTo, isNull: true)
}

/**
  Add a constraint that requires a particular key to not be equal to **undefined**.
  - parameter key: The key that should exist.
  - returns: The resulting `QueryConstraint`.
 */
public func exists(key: String) -> QueryConstraint {
    .init(key: key, value: true, comparator: .exists)
}

/**
  Add a constraint that requires a key to be equal to **undefined**.
  - parameter key: The key that should not exist.
  - returns: The resulting `QueryConstraint`.
 */
public func doesNotExist(key: String) -> QueryConstraint {
    .init(key: key, value: false, comparator: .exists)
}

Example use cases are in Swift Playgrounds: https://github.com/parse-community/Parse-Swift/blob/3d4bb13acd7496a49b259e541928ad493219d363/ParseSwift.playground/Pages/13%20-%20Operations.xcplaygroundpage/Contents.swift#L92-L135