Skip to content

Sort broken using MultiIndex with mixed ascending #16934

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
alubbock opened this issue Jul 14, 2017 · 3 comments · Fixed by #16937
Closed

Sort broken using MultiIndex with mixed ascending #16934

alubbock opened this issue Jul 14, 2017 · 3 comments · Fixed by #16937
Labels
Bug MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@alubbock
Copy link
Contributor

alubbock commented Jul 14, 2017

Code Sample, a copy-pastable example if possible

# Imports
import pandas as pd

# Set up a Series with a three level MultiIndex
arrays = [['bar', 'bar', 'baz', 'baz', 'foo', 'foo', 'qux', 'qux'],
  ['one', 'two', 'one', 'two', 'one', 'two', 'one', 'two'],
  [4, 3, 2, 1, 4, 3, 2, 1]]
tuples = list(zip(*arrays))
index = pd.MultiIndex.from_tuples(tuples, names=['first', 'second', 'third'])
s = pd.Series(range(8), index=index)

# Sort descending works
s.sort_index(level=['third', 'first'], ascending=False)

# But sort descending on third level, ascending on first does not
s.sort_index(level=['third', 'first'], ascending=[False, True])

Problem description

Sorting using a list for the ascending argument appears to have been implemented in #10905 but doesn't work in the above scenario. I get the following output:

>>> s.sort_index(level=['third', 'first'], ascending=[False, True])

first  second  third
qux    one     2        6
       two     1        7
foo    one     4        4
       two     3        5
baz    one     2        2
       two     1        3
bar    one     4        0
       two     3        1
dtype: int64

Expected Output

DataFrame sorted by index third descending then index first ascending:

>>> s.sort_index(level=['third', 'first'], ascending=[False, True])

first  second  third
bar    one     4        0
foo    one     4        4
bar    two     3        1
foo    two     3        5
baz    one     2        2
qux    one     2        6
baz    two     1        3
qux    two     1        7

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.0.final.0
python-bits: 64
OS: Darwin
OS-release: 16.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.20.2
pytest: None
pip: 9.0.1
setuptools: 35.0.1
Cython: None
numpy: 1.13.0
scipy: 0.19.0
xarray: None
IPython: 6.1.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: 0.9.6
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None

@gfyoung
Copy link
Member

gfyoung commented Jul 15, 2017

@alubbock : Would you mind adding (with a hand-constructed DataFrame) what you expect the output to look like? That would be useful in addition to your description.

Otherwise, this definitely looks like a bug. Nice catch! PR is welcome.

cc @roblevy

@alubbock
Copy link
Contributor Author

@gfyoung I've edited the issue to add expected output and removed the np.random call to make the example deterministic. Thanks.

@gfyoung
Copy link
Member

gfyoung commented Jul 15, 2017

@alubbock : Thanks! Feel free to submit a PR for this if you know where the issue is.

alubbock added a commit to alubbock/pandas that referenced this issue Jul 15, 2017
MultiIndex sorting with `sort_index` would fail when the `ascending`
argument was specified as a list but not all levels of the index were
specified in the `level` argument, or the levels were specified in
a different order to the MultiIndex.

This PR rectifies the issue and introduces a unit test based on pandas-dev#16934

Fixes: pandas-dev#16934
alubbock added a commit to alubbock/pandas that referenced this issue Jul 15, 2017
MultiIndex sorting with `sort_index` would fail when the `ascending`
argument was specified as a list but not all levels of the index were
specified in the `level` argument, or the levels were specified in
a different order to the MultiIndex.

This PR rectifies the issue and introduces a unit test based on pandas-dev#16934

Fixes: pandas-dev#16934
alubbock added a commit to alubbock/pandas that referenced this issue Jul 15, 2017
MultiIndex sorting with `sort_index` would fail when the `ascending`
argument was specified as a list but not all levels of the index were
specified in the `level` argument, or the levels were specified in
a different order to the MultiIndex.

This PR rectifies the issue and introduces a unit test based on pandas-dev#16934

Fixes: pandas-dev#16934
@jreback jreback added this to the 0.21.0 milestone Jul 15, 2017
@jreback jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Jul 15, 2017
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Bug MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants