diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..dc88393 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,10 @@ +# gneiss changelog + +## Version 0.0.2 (changes since 0.0.2 go here) + +### Features +* * Adding in a niche sorting algorithm `gneiss.sort.niche_sort` that can generate a band table given a gradient [#16](https://github.com/biocore/gneiss/pull/16) +* Adding in utility functions for handing feature tables, metadata, and trees. [#12](https://github.com/biocore/gneiss/pull/12) +* Adding GPL license. + +### Bug fixes diff --git a/COPYING.txt b/COPYING.txt new file mode 100644 index 0000000..94a9ed0 --- /dev/null +++ b/COPYING.txt @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/gneiss/__init__.py b/gneiss/__init__.py index ccb3281..a27c03f 100644 --- a/gneiss/__init__.py +++ b/gneiss/__init__.py @@ -1,7 +1,7 @@ # ---------------------------------------------------------------------------- # Copyright (c) 2016--, gneiss development team. # -# Distributed under the terms of the Modified BSD License. +# Distributed under the terms of the GPLv3 License. # # The full license is in the file COPYING.txt, distributed with this software. # ---------------------------------------------------------------------------- @@ -9,4 +9,4 @@ from __future__ import absolute_import, division, print_function -__version__ = "0.0.1" +__version__ = "0.0.2" diff --git a/gneiss/layouts.py b/gneiss/layouts.py index 56d88ad..a831cba 100644 --- a/gneiss/layouts.py +++ b/gneiss/layouts.py @@ -1,3 +1,11 @@ +# ---------------------------------------------------------------------------- +# Copyright (c) 2016--, gneiss development team. +# +# Distributed under the terms of the Modified BSD License. +# +# The full license is in the file COPYING.txt, distributed with this software. +# ---------------------------------------------------------------------------- + from ete3 import faces, AttrFace, CircleFace, BarChartFace diff --git a/gneiss/sort.py b/gneiss/sort.py new file mode 100644 index 0000000..70cdc58 --- /dev/null +++ b/gneiss/sort.py @@ -0,0 +1,116 @@ +# ---------------------------------------------------------------------------- +# Copyright (c) 2016--, gneiss development team. +# +# Distributed under the terms of the GPLv3 License. +# +# The full license is in the file COPYING.txt, distributed with this software. +# ---------------------------------------------------------------------------- +import numpy as np +import pandas as pd +from functools import partial +from gneiss.util import match + + +def mean_niche_estimator(abundances, gradient): + """ Estimates the mean niche of an organism. + + Calculates the mean niche of an organism along a gradient. + This is done by calculating the expected value of an organism + across the gradient. + + Specifically, this module calculates the following + + .. math:: + E[g | x] = + \sum\limits_{i=1}^N g_i \frac{x_i}{\sum\limits_{j=1}^N x_j} + + Where :math:`N` is the number of samples, :math:`x_i` is the proportion of + species :math:`x` in sample :math:`i`, :math:`g_i` is the gradient value + at sample `i`. + + Parameters + ---------- + abundances : pd.Series, np.float + Vector of fraction abundances of an organism over + a list of samples. + gradient : pd.Series, np.float + Vector of numerical gradient values. + + Returns + ------- + np.float : + The mean gradient that the organism lives in. + + Raises + ------ + ValueError: + If the length of `abundances` is not the same length as `gradient`. + ValueError: + If the length of `gradient` contains nans. + """ + len_abundances = len(abundances) + len_gradient = len(gradient) + if len_abundances != len_gradient: + raise ValueError("Length of `abundances` (%d) doesn't match the length" + " of the `gradient` (%d)" % (len_abundances, + len_gradient)) + if np.any(pd.isnull(gradient)): + raise ValueError("`gradient` cannot have any nans.") + + # normalizes the proportions of the organism across all of the + # samples to add to 1. + v = abundances / abundances.sum() + m = np.dot(gradient, v) + return m + + +def niche_sort(table, gradient, niche_estimator=mean_niche_estimator): + """ Sort the table according to estimated niches. + + Sorts the table by samples along the gradient + and otus by their estimated niche along the gradient. + + Parameters + ---------- + table : pd.DataFrame + Contingency table where samples are rows and + features (i.e. OTUs) are columns. + gradient : pd.Series + Vector of numerical gradient values. + niche_estimator : function, optional + A function that takes in two pandas series and returns an ordered + object. The ability for the object to be ordered is critical, since + this will allow the table to be sorted according to this ordering. + By default, `mean_niche_estimator` will be used. + + Returns + ------- + pd.DataFrame : + Sorted table according to the gradient of the samples, and the niches + of the organisms along that gradient. + + Raises + ------ + ValueError : + Raised if `niche_estimator` is not a function. + """ + if not callable(niche_estimator): + raise ValueError("`niche_estimator` is not a function.") + + table, gradient = match(table, gradient) + + niche_estimator = partial(niche_estimator, + gradient=gradient) + + # normalizes feature abundances to sum to 1, for each sample. + # (i.e. scales values in each row to sum to 1). + normtable = table.apply(lambda x: x/x.sum(), axis=1) + + # calculates estimated niche for each feature + est_niche = normtable.apply(niche_estimator, axis=0) + gradient = gradient.sort_values() + est_niche = est_niche.sort_values() + + table = table.reindex(index=gradient.index, + columns=est_niche.index) + return table diff --git a/gneiss/tests/data/large_tree2.nwk b/gneiss/tests/data/large_tree2.nwk new file mode 100644 index 0000000..a1fb1c7 --- /dev/null +++ b/gneiss/tests/data/large_tree2.nwk @@ -0,0 +1 @@ +(O0:0.49383032675,((((((((((((((O614:3.48384118094e-12,O615:3.48384118094e-12):1.22959004914e-11,(O612:3.52949290341e-12,O613:3.52949290341e-12):1.2250248769e-11):4.99466671188e-11,((O610:3.57589484816e-12,O611:3.57589484816e-12):1.26214873337e-11,(O608:3.62306185855e-12,O609:3.62306185855e-12):1.25743203233e-11):4.95290266093e-11):2.06028942932e-10,(((O606:3.67100912172e-12,O607:3.67100912172e-12):1.29579225153e-11,(O604:3.7197521777e-12,O605:3.7197521777e-12):1.29091794594e-11):5.26465662255e-11,((O602:3.7693069288e-12,O603:3.7693069288e-12):1.33056426928e-11,(O600:3.8196896493e-12,O601:3.8196896493e-12):1.32552599723e-11):5.22005482409e-11):2.02479853861e-10):8.7706467622e-10,((((O598:3.87091699546e-12,O599:3.87091699546e-12):1.36651051662e-11,(O596:3.92300601575e-12,O597:3.92300601575e-12):1.3613016146e-11):5.55313871705e-11,((O594:3.97597416146e-12,O595:3.97597416146e-12):1.4036789002e-11,(O592:4.02983929764e-12,O593:4.02983929764e-12):1.39829238658e-11):5.50546461687e-11):2.2925627493e-10,(((O590:4.08461971432e-12,O591:4.08461971432e-12):1.44211962249e-11,(O588:4.1403341381e-12,O589:4.1403341381e-12):1.43654818011e-11):5.86165391665e-11,((O586:4.19700174412e-12,O587:4.19700174412e-12):1.48188530845e-11,(O584:4.25464216838e-12,O585:4.25464216838e-12):1.47612126602e-11):5.8106500277e-11):2.25201329156e-10):8.46496343682e-10):3.98769083028e-09,(((((O582:4.31327552042e-12,O583:4.31327552042e-12):1.52303114e-11,(O580:4.37292239643e-12,O581:4.37292239643e-12):1.5170664524e-11):6.19189504788e-11,((O578:4.43360389277e-12,O579:4.43360389277e-12):1.56561499903e-11,(O576:4.49534161987e-12,O577:4.49534161987e-12):1.55944122632e-11):6.13727835162e-11):2.55851125973e-10,(((O574:4.55815771663e-12,O575:4.55815771663e-12):1.60969761939e-11,(O572:4.62207486521e-12,O573:4.62207486521e-12):1.60330590453e-11):6.54572420036e-11,((O570:4.68711630633e-12,O571:4.68711630633e-12):1.65534274869e-11,(O568:4.75330585506e-12,O569:4.75330585506e-12):1.64872379382e-11):6.48718321209e-11):2.51201287458e-10):1.09298684219e-09,((((O566:4.82066791708e-12,O567:4.82066791708e-12):1.70261732038e-11,(O564:4.88922750549e-12,O565:4.88922750549e-12):1.69576136154e-11):6.9251923482e-11,((O562:4.95901025813e-12,O563:4.95901025813e-12):1.75159163705e-11,(O560:5.03004245551e-12,O561:5.03004245551e-12):1.74448841731e-11):6.86238379743e-11):2.86416735085e-10,(((O558:5.10235103927e-12,O559:5.10235103927e-12):1.80233956561e-11,(O556:5.17596363124e-12,O557:5.17596363124e-12):1.79497830641e-11):7.33256156785e-11,((O554:5.25090855315e-12,O555:5.25090855315e-12):1.85493874536e-11,(O552:5.32721484701e-12,O553:5.32721484701e-12):1.84730811598e-11):7.26510663671e-11):2.81064137314e-10):1.05278500588e-09):3.70621035266e-09):2.09465786679e-08,((((((O550:5.40491229603e-12,O551:5.40491229603e-12):1.90947080975e-11,(O548:5.48403144642e-12,O549:5.48403144642e-12):1.90155889471e-11):7.77033023964e-11,((O546:5.56460362975e-12,O547:5.56460362975e-12):1.96602162298e-11,(O544:5.6466609861e-12,O545:5.6466609861e-12):1.95781588734e-11):7.69781029304e-11):3.21686263099e-10,(((O542:5.73023648807e-12,O543:5.73023648807e-12):2.02468153241e-11,(O540:5.81536396541e-12,O541:5.81536396541e-12):2.01616878468e-11):8.24126168517e-11,((O538:5.90207813068e-12,O539:5.90207813068e-12):2.08554563814e-11,(O536:5.99041460561e-12,O537:5.99041460561e-12):2.07671199065e-11):8.16321341518e-11):3.15499517225e-10):1.37964219505e-09,((((O534:6.08040994844e-12,O535:6.08040994844e-12):2.14871408084e-11,(O532:6.17210168217e-12,O533:6.17210168217e-12):2.13954490747e-11):8.7484167688e-11,((O530:6.26552832375e-12,O531:6.26552832375e-12):2.21429234946e-11,(O528:6.3607294143e-12,O529:6.3607294143e-12):2.20477224041e-11):8.66432666265e-11):3.62555779095e-10,(((O526:6.45774555037e-12,O527:6.45774555037e-12):2.28239161016e-11,(O524:6.55661841618e-12,O525:6.55661841618e-12):2.27250432358e-11):9.29519108254e-11,((O522:6.65739081717e-12,O523:6.65739081717e-12):2.35312905819e-11,(O520:6.76010671448e-12,O521:6.76010671448e-12):2.34285746846e-11):9.20448910784e-11):3.55373925062e-10):1.3259238834e-09):6.37273122717e-09,(((((O518:6.86481126086e-12,O519:6.86481126086e-12):2.42662829448e-11,(O516:6.97155083762e-12,O517:6.97155083762e-12):2.41595433681e-11):9.8853574416e-11,((O514:7.08037309307e-12,O515:7.08037309307e-12):2.503019729e-11,(O512:7.19132698218e-12,O513:7.19132698218e-12):2.49192434009e-11):9.78740982386e-11):4.10126975163e-10,(((O510:7.30446280771e-12,O511:7.30446280771e-12):2.58244101275e-11,(O508:7.41983226279e-12,O509:7.41983226279e-12):2.57090406724e-11):1.05231145468e-10,((O506:7.53748847499e-12,O507:7.53748847499e-12):2.66503750098e-11,(O504:7.65748605202e-12,O505:7.65748605202e-12):2.65303774328e-11):1.04172154918e-10):4.01751625382e-10):1.76675166338e-09,((((O502:7.77988112901e-12,O503:7.77988112901e-12):2.75096274994e-11,(O500:7.90473141759e-12,O501:7.90473141759e-12):2.73847772109e-11):1.12131428247e-10,((O498:8.03209625664e-12,O499:8.03209625664e-12):2.84037904999e-11,(O496:8.16203666495e-12,O497:8.16203666495e-12):2.82738500916e-11):1.10985050118e-10):4.65762904455e-10,(((O494:8.2946153958e-12,O495:8.2946153958e-12):2.93345799795e-11,(O492:8.42989699356e-12,O493:8.42989699356e-12):2.91992983817e-11):1.19606686405e-10,((O490:8.56794785231e-12,O491:8.56794785231e-12):3.0303811121e-11,(O488:8.70883627668e-12,O489:8.70883627668e-12):3.01629226966e-11):1.18364122807e-10):4.5594795955e-10):1.69367946583e-09):5.86939930094e-09):1.7906826918e-08):1.52270804628e-07,((((((((O742:1.63623811764e-12,O743:1.63623811764e-12):5.76663074192e-12,(O740:1.65396295386e-12,O741:1.65396295386e-12):5.7489059057e-12):2.33621300773e-11,((O738:1.67192845065e-12,O739:1.67192845065e-12):5.89263597425e-12,(O736:1.6901385398e-12,O737:1.6901385398e-12):5.8744258851e-12):2.32004345119e-11):9.58541951167e-11,(((O734:1.70859722823e-12,O735:1.70859722823e-12):6.02210172218e-12,(O732:1.72730859964e-12,O733:1.72730859964e-12):6.00339035076e-12):2.44005034965e-11,((O730:1.74627681624e-12,O731:1.74627681624e-12):6.15514265922e-12,(O728:1.76550612043e-12,O729:1.76550612043e-12):6.13591335503e-12):2.42297829715e-11):9.44879916066e-11):4.03606633581e-10,((((O726:1.78500083659e-12,O727:1.78500083659e-12):6.29187791684e-12,(O724:1.80476537288e-12,O725:1.80476537288e-12):6.27211338055e-12):2.54972139585e-11,((O722:1.82480422312e-12,O723:1.82480422312e-12):6.43243128359e-12,(O720:1.84512196872e-12,O721:1.84512196872e-12):6.412113538e-12):2.53168572052e-11):1.04673473794e-10,(((O718:1.86572328056e-12,O719:1.86572328056e-12):6.57693141438e-12,(O716:1.88661292106e-12,O717:1.88661292106e-12):6.55604177387e-12):2.66562389669e-11,((O714:1.90779574624e-12,O715:1.90779574624e-12):6.72551205029e-12,(O712:1.92927670777e-12,O713:1.92927670777e-12):6.70403108876e-12):2.64655858653e-11):1.03148672844e-10):3.91978261129e-10):1.79318851436e-09,(((((O710:1.95106085517e-12,O711:1.95106085517e-12):6.87831224974e-12,(O708:1.97315333803e-12,O709:1.97315333803e-12):6.85621976687e-12):2.78818760429e-11,((O706:1.99555940828e-12,O707:1.99555940828e-12):7.03547663142e-12,(O704:2.01828442249e-12,O705:2.01828442249e-12):7.01275161722e-12):2.76802131081e-11):1.14530792566e-10,(((O702:2.04133384429e-12,O703:2.04133384429e-12):7.19715562988e-12,(O700:2.06471324683e-12,O701:2.06471324683e-12):7.17377622735e-12):2.91787724916e-11,((O698:2.08842831526e-12,O699:2.08842831526e-12):7.36350576435e-12,(O696:2.11248484936e-12,O697:2.11248484936e-12):7.33944923025e-12):2.89653278862e-11):1.12824779748e-10):4.83386373511e-10,((((O694:2.13688876613e-12,O695:2.13688876613e-12):7.53468992162e-12,(O692:2.16164610256e-12,O693:2.16164610256e-12):7.5099325852e-12):3.05519583209e-11,((O690:2.18676301837e-12,O691:2.18676301837e-12):7.71087765387e-12,(O688:2.21224579893e-12,O689:2.21224579893e-12):7.68539487332e-12):3.03258963364e-11):1.25576279623e-10,(((O686:2.23810085812e-12,O687:2.23810085812e-12):7.89224549218e-12,(O684:2.26433474141e-12,O685:2.26433474141e-12):7.86601160888e-12):3.20068826863e-11,((O682:2.29095412894e-12,O683:2.29095412894e-12):8.07897727676e-12,(O680:2.31796583865e-12,O681:2.31796583865e-12):8.05196556705e-12):3.17672976309e-11):1.23662587595e-10):4.68828598594e-10):1.68878592677e-09):8.94425570194e-09,((((((O678:2.34537682959e-12,O679:2.34537682959e-12):8.27126450489e-12,(O676:2.37319420527e-12,O677:2.37319420527e-12):8.24344712921e-12):3.35494542873e-11,((O674:2.40142521705e-12,O675:2.40142521705e-12):8.46930669756e-12,(O672:2.4300772677e-12,O673:2.4300772677e-12):8.44065464691e-12):3.32953637072e-11):1.37986016359e-10,(((O670:2.45915791503e-12,O671:2.45915791503e-12):8.67331178603e-12,(O668:2.48867487559e-12,O669:2.48867487559e-12):8.64379482547e-12):3.51860861975e-11,((O666:2.51863602851e-12,O667:2.51863602851e-12):8.88349651939e-12,(O664:2.54904941943e-12,O665:2.54904941943e-12):8.85308312848e-12):3.49164233506e-11):1.35833556082e-10):5.83889978139e-10,((((O662:2.57992326449e-12,O663:2.57992326449e-12):9.10008689456e-12,(O660:2.61126595453e-12,O661:2.61126595453e-12):9.06874420452e-12):3.69237456741e-11,((O658:2.64308605933e-12,O659:2.64308605933e-12):9.32331860999e-12,(O656:2.67539233196e-12,O657:2.67539233196e-12):9.29101233736e-12):3.66373511638e-11):1.51967293989e-10,(((O654:2.70819371331e-12,O655:2.70819371331e-12):9.55343754451e-12,(O652:2.74149933667e-12,O653:2.74149933667e-12):9.52013192115e-12):3.87700095716e-11,((O650:2.7753185325e-12,O651:2.7753185325e-12):9.79070026302e-12,(O648:2.80966083334e-12,O649:2.80966083334e-12):9.75635796218e-12):3.84656220339e-11):1.49539408993e-10):5.65471040297e-10):2.62127907462e-09,(((((O646:2.84453597875e-12,O647:2.84453597875e-12):1.00353745506e-11,(O644:2.87995392055e-12,O645:2.87995392055e-12):9.99995660877e-12):4.07331260665e-11,((O642:2.91592482806e-12,O643:2.91592482806e-12):1.02877399766e-11,(O640:2.95245909357e-12,O641:2.95245909357e-12):1.02512057111e-11):4.04093717912e-11):1.67765098848e-10,(((O638:2.98956733795e-12,O639:2.98956733795e-12):1.05480884915e-11,(O636:3.02726041643e-12,O637:3.02726041643e-12):1.0510395413e-11):4.28220835038e-11,((O634:3.06554942448e-12,O635:3.06554942448e-12):1.0816725057e-11,(O632:3.10444570396e-12,O633:3.10444570396e-12):1.07778287775e-11):4.24774648517e-11):1.65018396111e-10):7.11926621383e-10,((((O630:3.14396084935e-12,O631:3.14396084935e-12):1.10939683134e-11,(O628:3.18410671423e-12,O629:3.18410671423e-12):1.10538224485e-11):4.50466872886e-11,((O626:3.22489541791e-12,O627:3.22489541791e-12):1.1380151285e-11,(O624:3.26633935224e-12,O625:3.26633935224e-12):1.13387073507e-11):4.46795697485e-11):1.85670153935e-10,(((O622:3.30845118869e-12,O623:3.30845118869e-12):1.16756221272e-11,(O620:3.3512438855e-12,O621:3.3512438855e-12):1.16328294304e-11):4.74176458829e-11,((O618:3.39473069521e-12,O619:3.39473069521e-12):1.19807449171e-11,(O616:3.43892517227e-12,O617:3.43892517227e-12):1.19365504401e-11):4.70262435864e-11):1.82553051188e-10):6.88349986441e-10):2.45401640791e-09):7.8803488792e-09):4.78304548106e-08,((((((((O998:5.01001502002e-13,O999:5.01001502002e-13):1.76255898739e-12,(O996:5.0503166072e-13,O997:5.0503166072e-13):1.75852882867e-12):7.1172645738e-12,((O994:5.09102425336e-13,O995:5.09102425336e-13):1.7910957081e-12,(O992:5.13214287792e-13,O993:5.13214287792e-13):1.78698384565e-12):7.08062692975e-12):2.90110750381e-11,(((O990:5.17367746998e-13,O991:5.17367746998e-13):1.82021230357e-12,(O988:5.2156330894e-13,O989:5.2156330894e-13):1.81601674162e-12):7.35063327763e-12,((O986:5.25801486803e-13,O987:5.25801486803e-13):1.84992297129e-12,(O984:5.30082801083e-13,O985:5.30082801083e-13):1.84564165701e-12):7.3124888701e-12):2.87036867731e-11):1.2053463526e-10,((((O982:5.34407779707e-13,O983:5.34407779707e-13):1.880242316e-12,(O980:5.38776958159e-13,O981:5.38776958159e-13):1.87587313755e-12):7.59364581631e-12,((O978:5.43190879594e-13,O979:5.43190879594e-13):1.91118536302e-12,(O976:5.47650094977e-13,O977:5.47650094977e-13):1.90672614764e-12):7.5539196694e-12):3.0962447797e-11,(((O974:5.52155163198e-13,O975:5.52155163198e-13):1.94276757219e-12,(O972:5.56706651215e-13,O973:5.56706651215e-13):1.93821608417e-12):7.84678439883e-12,((O970:5.61305134178e-13,O971:5.61305134178e-13):1.97500485226e-12,(O968:5.65951195568e-13,O969:5.65951195568e-13):1.97035879087e-12):7.80539714778e-12):3.06290365748e-11):1.17955791652e-10):5.20837433208e-10,(((((O966:5.70645427338e-13,O967:5.70645427338e-13):2.00791357589e-12,(O964:5.75388430048e-13,O965:5.75388430048e-13):2.00317057318e-12):8.11055959994e-12,((O962:5.80180813014e-13,O963:5.80180813014e-13):2.04151059514e-12,(O960:5.8502319445e-13,O961:5.8502319445e-13):2.0366682137e-12):8.06742719502e-12):3.30806841384e-11,(((O958:5.8991620162e-13,O959:5.8991620162e-13):2.07581325757e-12,(O956:5.94860470987e-13,O957:5.94860470987e-13):2.0708689882e-12):8.38551228328e-12,((O954:5.9985664837e-13,O955:5.9985664837e-13):2.11083942302e-12,(O952:6.04905389098e-13,O953:6.04905389098e-13):2.10579068229e-12):8.34054567108e-12):3.27185609991e-11):1.37618494993e-10,((((O950:6.10007358175e-13,O951:6.10007358175e-13):2.1466074809e-12,(O948:6.15163230441e-13,O949:6.15163230441e-13):2.14145160864e-12):8.6722156724e-12,((O946:6.20373690738e-13,O947:6.20373690738e-13):2.18313636827e-12,(O944:6.2563943408e-13,O945:6.2563943408e-13):2.17787062493e-12):8.62532045247e-12):3.53832031149e-11,(((O942:6.30961165828e-13,O943:6.30961165828e-13):2.22044558852e-12,(O940:6.36339601864e-13,O941:6.36339601864e-13):2.21506715249e-12):8.97127758253e-12,((O938:6.41775468774e-13,O939:6.41775468774e-13):2.25855523084e-12,(O936:6.47269504027e-13,O937:6.47269504027e-13):2.25306119558e-12):8.92235363727e-12):3.49893492895e-11):1.34576264108e-10):4.98375670835e-10):2.44491067854e-09,((((((O934:6.52822456163e-13,O935:6.52822456163e-13):2.29748599036e-12,(O932:6.58435084987e-13,O933:6.58435084987e-13):2.29187336154e-12):9.28334282687e-12,((O930:6.64108161759e-13,O931:6.64108161759e-13):2.33725918922e-12,(O928:6.69842469391e-13,O929:6.69842469391e-13):2.33152488158e-12):9.23228392242e-12):3.78895826393e-11,(((O926:6.75638802655e-13,O927:6.75638802655e-13):2.37789679827e-12,(O924:6.81497968381e-13,O925:6.81497968381e-13):2.37203763255e-12):9.60909581302e-12,((O922:6.87420785672e-13,O923:6.87420785672e-13):2.41942145982e-12,(O920:6.93408086115e-13,O921:6.93408086115e-13):2.41343415938e-12):9.55578916845e-12):3.74606024988e-11):1.57839245696e-10,((((O918:6.99460713999e-13,O919:6.99460713999e-13):2.46185651113e-12,(O916:7.0557952654e-13,O917:7.0557952654e-13):2.45573769859e-12):9.94926334602e-12,((O914:7.11765394104e-13,O915:7.11765394104e-13):2.5052260089e-12,(O912:7.18019200441e-13,O913:7.18019200441e-13):2.49897220257e-12):9.89358916814e-12):4.06218707575e-11,(((O910:7.24341842919e-13,O911:7.24341842919e-13):2.54955475478e-12,(O908:7.30734232765e-13,O909:7.30734232765e-13):2.54316236493e-12):1.03046176568e-11,((O906:7.37197295308e-13,O907:7.37197295308e-13):2.59486832178e-12,(O904:7.43731970235e-13,O905:7.43731970235e-13):2.58833364685e-12):1.02464486374e-11):4.01539370741e-11):1.5423002828e-10):6.85775863887e-10,(((((O902:7.50339211838e-13,O903:7.50339211838e-13):2.64119308187e-12,(O900:7.57019989281e-13,O901:7.57019989281e-13):2.63451230443e-12):1.06759796761e-11,((O898:7.63775286862e-13,O899:7.63775286862e-13):2.6885562346e-12,(O896:7.70606104283e-13,O897:7.70606104283e-13):2.68172541718e-12):1.06151804483e-11):4.36049483345e-11,(((O894:7.77513456929e-13,O895:7.77513456929e-13):2.73698583692e-12,(O892:7.84498376148e-13,O893:7.84498376148e-13):2.73000091771e-12):1.10642225761e-11,((O890:7.91561909537e-13,O891:7.91561909537e-13):2.78651083423e-12,(O888:7.98705121239e-13,O889:7.98705121239e-13):2.77936762252e-12):1.10006491262e-11):4.30937384343e-11):1.81914176356e-10,((((O886:8.05929092239e-13,O887:8.05929092239e-13):2.83716109262e-12,(O884:8.13234920672e-13,O885:8.13234920672e-13):2.82985526419e-12):1.14702756048e-11,((O882:8.20623722132e-13,O883:8.20623722132e-13):2.88896743258e-12,(O880:8.28096629992e-13,O881:8.28096629992e-13):2.88149452472e-12):1.1403774635e-11):4.68669527645e-11,(((O878:8.35654795726e-13,O879:8.35654795726e-13):2.94196166394e-12,(O876:8.43299389245e-13,O877:8.43299389245e-13):2.93431707042e-12):1.18951282397e-11,((O874:8.51031599229e-13,O875:8.51031599229e-13):2.99617662236e-12,(O872:8.58852633476e-13,O873:8.58852633476e-13):2.98835558812e-12):1.18255364778e-11):4.63075738548e-11):1.77606318106e-10):6.54151706836e-10):2.23093630361e-09):1.38276771e-08,(((((((O870:8.66763719255e-13,O871:8.66763719255e-13):3.05164620733e-12,(O868:8.74766103661e-13,O869:8.74766103661e-13):3.04364382292e-12):1.23398346908e-11,((O866:8.82861053987e-13,O867:8.82861053987e-13):3.1084054217e-12,(O864:8.91049858094e-13,O865:8.91049858094e-13):3.10021661759e-12):1.22669781417e-11):5.04397743307e-11,(((O862:8.99333824798e-13,O863:8.99333824798e-13):3.16649041295e-12,(O860:9.07714284256e-13,O861:9.07714284256e-13):3.15810995349e-12):1.28055187858e-11,((O858:9.1619258837e-13,O859:9.1619258837e-13):3.22593851616e-12,(O856:9.24770111188e-13,O857:9.24770111188e-13):3.21736099334e-12):1.27292119191e-11):4.98266759244e-11):2.107594371e-10,((((O854:9.33448249326e-13,O855:9.33448249326e-13):3.2867882988e-12,(O852:9.42228422386e-13,O853:9.42228422386e-13):3.27800812574e-12):1.32933792749e-11,((O850:9.51112073394e-13,O851:9.51112073394e-13):3.34907960747e-12,(O848:9.60100669243e-13,O849:9.60100669243e-13):3.34009101162e-12):1.32134241422e-11):5.43596393215e-11,(((O846:9.69195701143e-13,O847:9.69195701143e-13):3.41285361658e-12,(O844:9.78398685081e-13,O845:9.78398685081e-13):3.40365063264e-12):1.38046955931e-11,((O842:9.87711162299e-13,O843:9.87711162299e-13):3.47815287917e-12,(O840:9.97134699769e-13,O841:9.97134699769e-13):3.4687293417e-12):1.37208808694e-11):5.36865102337e-11):2.05584200903e-10):9.21497104169e-10,(((((O838:1.00667089069e-12,O839:1.00667089069e-12):3.54502137993e-12,(O836:1.01632135499e-12,O837:1.01632135499e-12):3.53537091563e-12):1.43408341275e-11,((O834:1.02608773983e-12,O835:1.02608773983e-12):3.6135045905e-12,(O832:1.03597172016e-12,O833:1.03597172016e-12):3.60362061018e-12):1.42529340678e-11):5.86677970859e-11,(((O830:1.0459749992e-12,O831:1.0459749992e-12):3.68364952722e-12,(O828:1.05609930904e-12,O829:1.05609930904e-12):3.67352521737e-12):1.49032550365e-11,((O826:1.06634641119e-12,O827:1.06634641119e-12):3.75550481135e-12,(O824:1.07671809713e-12,O825:1.07671809713e-12):3.74513312541e-12):1.48110283404e-11):5.79274439212e-11):2.45555619674e-10,((((O822:1.0872161889e-12,O823:1.0872161889e-12):3.82912073202e-12,(O820:1.09784253972e-12,O821:1.09784253972e-12):3.8184943812e-12):1.54935196772e-11,((O818:1.10859903457e-12,O819:1.10859903457e-12):3.90454931194e-12,(O816:1.11948759084e-12,O817:1.11948759084e-12):3.89366075567e-12):1.53967082516e-11):6.34113320253e-11,(((O814:1.13051015897e-12,O815:1.13051015897e-12):3.98184437603e-12,(O812:1.14166872306e-12,O813:1.14166872306e-12):3.97068581194e-12):1.61132987013e-11,((O810:1.1529653016e-12,O811:1.1529653016e-12):4.0610616231e-12,(O808:1.16440194808e-12,O809:1.16440194808e-12):4.04962497662e-12):1.60116263116e-11):6.25955353872e-11):2.39294754535e-10):8.75838617058e-10):4.4373966765e-09,((((((O806:1.17598075175e-12,O807:1.17598075175e-12):4.14225870083e-12,(O804:1.18770383832e-12,O805:1.18770383832e-12):4.13053561426e-12):1.67643808861e-11,((O802:1.19957337064e-12,O803:1.19957337064e-12):4.22549528405e-12,(O800:1.21159154952e-12,O801:1.21159154952e-12):4.21347710517e-12):1.6657551684e-11):6.86441262844e-11,(((O798:1.22376061441e-12,O799:1.22376061441e-12):4.31083315663e-12,(O796:1.23608284426e-12,O797:1.23608284426e-12):4.29851092678e-12):1.74486827779e-11,((O794:1.24856055825e-12,O795:1.24856055825e-12):4.39833629713e-12,(O792:1.26119611667e-12,O793:1.26119611667e-12):4.38570073872e-12):1.73363796936e-11):6.77434700741e-11):2.87838703563e-10,((((O790:1.27399192166e-12,O791:1.27399192166e-12):4.48807096829e-12,(O788:1.28695041817e-12,O789:1.28695041817e-12):4.47511247178e-12):1.81682592286e-11,((O786:1.30007409477e-12,O787:1.30007409477e-12):4.58010581076e-12,(O784:1.31336548453e-12,O785:1.31336548453e-12):4.566814421e-12):1.8050142213e-11):7.44280048675e-11,(((O782:1.32682716597e-12,O783:1.32682716597e-12):4.67451194111e-12,(O780:1.340461764e-12,O781:1.340461764e-12):4.66087734308e-12):1.8925314921e-11,((O778:1.35427195082e-12,O779:1.35427195082e-12):4.7713630544e-12,(O776:1.36826044695e-12,O777:1.36826044695e-12):4.75737455826e-12):1.88010190229e-11):7.34316729579e-11):2.802071232e-10):1.26779426489e-09,(((((O774:1.38243002222e-12,O775:1.38243002222e-12):4.87073553163e-12,(O772:1.39678349678e-12,O773:1.39678349678e-12):4.85638205707e-12):1.97222169859e-11,((O770:1.41132374215e-12,O771:1.41132374215e-12):4.97270855215e-12,(O768:1.42605368231e-12,O769:1.42605368231e-12):4.95797861199e-12):1.95913502455e-11):8.08341024129e-11,(((O766:1.44097629477e-12,O767:1.44097629477e-12):5.07736421149e-12,(O764:1.45609461171e-12,O765:1.45609461171e-12):5.06224589454e-12):2.05615088263e-11,((O762:1.47141172115e-12,O763:1.47141172115e-12):5.18478764472e-12,(O760:1.48693076807e-12,O761:1.48693076807e-12):5.16926859779e-12):2.04236499667e-11):7.97296356201e-11):3.39627753116e-10,((((O758:1.50265495569e-12,O759:1.50265495569e-12):5.29506715577e-12,(O756:1.51858754664e-12,O757:1.51858754664e-12):5.27913456482e-12):2.14459252803e-11,((O754:1.53473186425e-12,O755:1.53473186425e-12):5.40829435293e-12,(O752:1.55109129386e-12,O753:1.55109129386e-12):5.39193492332e-12):2.13006211746e-11):8.79445003385e-11,(((O750:1.56766928412e-12,O751:1.56766928412e-12):5.52456429095e-12,(O748:1.58446934834e-12,O749:1.58446934834e-12):5.50776422673e-12):2.23784092668e-11,((O746:1.60149506593e-12,O747:1.60149506593e-12):5.64397561999e-12,(O744:1.61875008375e-12,O745:1.61875008375e-12):5.62672060217e-12):2.2225172156e-11):8.67175048884e-11):3.30249090338e-10):1.19992247701e-09):3.98999152164e-09):1.13160005104e-08):4.21457731075e-08):1.19255769299e-07):1.18607212647e-06,(((((((O294:6.55756418701e-11,O295:6.55756418701e-11):2.33556267118e-10,(O292:6.73814112672e-11,O293:6.73814112672e-11):2.31750497721e-10):9.64839080615e-10,((O290:6.92497681113e-11,O291:6.92497681113e-11):2.46701415107e-10,(O288:7.11833336582e-11,O289:7.11833336582e-11):2.4476784956e-10):9.48019806384e-10):4.11825588627e-09,(((O286:7.31848581414e-11,O287:7.31848581414e-11):2.6078451917e-10,(O284:7.52572280767e-11,O285:7.52572280767e-11):2.58712149235e-10):1.07829870205e-09,((O282:7.74034740349e-11,O283:7.74034740349e-11):2.75887049995e-10,(O280:7.96267789166e-11,O281:7.96267789166e-11):2.73663745113e-10):1.05897755533e-09):3.96995879651e-09):1.88264491006e-08,((((O278:8.19304867667e-11,O279:8.19304867667e-11):2.92098855057e-10,(O276:8.43181121654e-11,O277:8.43181121654e-11):2.89711229658e-10):1.20893750104e-09,((O274:8.67933502407e-11,O275:8.67933502407e-11):3.09519157588e-10,(O272:8.9360087346e-11,O273:8.9360087346e-11):3.06952420483e-10):1.18665433503e-09):5.17988559203e-09,(((O270:9.20224124546e-11,O271:9.20224124546e-11):3.28257691581e-10,(O268:9.47846293226e-11,O269:9.47846293226e-11):3.25495474713e-10):1.35997357422e-09,((O266:9.76512694801e-11,O267:9.76512694801e-11):3.48435993734e-10,(O264:1.00627106113e-10,O265:1.00627106113e-10):3.45460157102e-10):1.33416641504e-09):4.98259875663e-09):1.74458235415e-08):1.00115430089e-07,(((((O262:1.03717168903e-10,O263:1.03717168903e-10):3.70188876828e-10,(O260:1.06926759902e-10,O261:1.06926759902e-10):3.66979285828e-10):1.53535080848e-09,((O258:1.10261470523e-10,O259:1.10261470523e-10):3.93666113288e-10,(O256:1.13727199722e-10,O257:1.13727199722e-10):3.90200384089e-10):1.5053292704e-09):6.6067672877e-09,(((O254:1.17330173491e-10,O255:1.17330173491e-10):4.1903436278e-10,(O252:1.2107696574e-10,O253:1.2107696574e-10):4.15287570531e-10):1.73993213319e-09,((O250:1.24974520695e-10,O251:1.24974520695e-10):4.46479383788e-10,(O248:1.29030176933e-10,O249:1.29030176933e-10):4.4242372755e-10):1.70484276497e-09):6.33972747246e-09):3.07288400863e-08,((((O246:1.33251693183e-10,O247:1.33251693183e-10):4.76208576429e-10,(O244:1.37647276047e-10,O245:1.37647276047e-10):4.71812993565e-10):1.97975263693e-09,((O242:1.42225609786e-10,O243:1.42225609786e-10):5.08453912548e-10,(O240:1.46995888364e-10,O241:1.46995888364e-10):5.0368363397e-10):1.9385333842e-09):8.5606359191e-09,(((O238:1.51967849925e-10,O239:1.51967849925e-10):5.43475319777e-10,(O236:1.57151813916e-10,O237:1.57151813916e-10):5.38291355786e-10):2.26235388188e-09,((O234:1.62558721087e-10,O235:1.62558721087e-10):5.81564599045e-10,(O232:1.68200176607e-10,O233:1.68200176607e-10):5.75923143524e-10):2.21367373145e-09):8.19205177406e-09):2.81950154026e-08):8.49792418368e-08):7.00739340796e-07,(((((((O358:2.99347824067e-11,O359:2.99347824067e-11):1.06285047433e-10,(O356:3.0610295267e-11,O357:3.0610295267e-11):1.05609534572e-10):4.36536822174e-10,((O354:3.1304970746e-11,O355:3.1304970746e-11):1.11167955201e-10,(O352:3.20194648755e-11,O353:3.20194648755e-11):1.10453461071e-10):4.30283726066e-10):1.84142763811e-09,(((O350:3.27544600401e-11,O351:3.27544600401e-11):1.16334522758e-10,(O348:3.35106661937e-11,O349:3.35106661937e-11):1.15578316605e-10):4.78102625938e-10,((O346:3.42888221399e-11,O347:3.42888221399e-11):1.21804756061e-10,(O344:3.50896968797e-11,O345:3.50896968797e-11):1.21003881321e-10):4.71098030536e-10):1.78699268139e-09):8.21114292469e-09,((((O342:3.59140910303e-11,O343:3.59140910303e-11):1.27600326732e-10,(O340:3.67628383194e-11,O341:3.67628383194e-11):1.26751579443e-10):5.24735573902e-10,((O338:3.76368071594e-11,O339:3.76368071594e-11):1.33744732456e-10,(O336:3.85369023067e-11,O337:3.85369023067e-11):1.32844637309e-10):5.16868452049e-10):2.21907752527e-09,(((O334:3.94640666098e-11,O335:3.94640666098e-11):1.40263474961e-10,(O332:4.04192828537e-11,O333:4.04192828537e-11):1.39308258717e-10):5.77195339803e-10,((O330:4.14035757047e-11,O331:4.14035757047e-11):1.47184257757e-10,(O328:4.24180137631e-11,O329:4.24180137631e-11):1.46169819699e-10):5.68335047911e-10):2.15040463556e-09):7.71799969788e-09):4.12850949571e-08,(((((O326:4.34637117284e-11,O327:4.34637117284e-11):1.54537206095e-10,(O324:4.45418326871e-11,O325:4.45418326871e-11):1.53459085137e-10):6.36377854649e-10,((O322:4.56535905273e-11,O323:4.56535905273e-11):1.62355111961e-10,(O320:4.68002524912e-11,O321:4.68002524912e-11):1.61208449997e-10):6.26370069984e-10):2.69870244945e-09,(((O318:4.79831418716e-11,O319:4.79831418716e-11):1.70673707305e-10,(O316:4.92036408636e-11,O317:4.92036408636e-11):1.69453208313e-10):7.0334396738e-10,((O314:5.04631935798e-11,O315:5.04631935798e-11):1.7953196918e-10,(O312:5.17633092405e-11,O313:5.17633092405e-11):1.78231853519e-10):6.92005653797e-10):2.61108040537e-09):1.21689026789e-08,((((O310:5.3105565551e-11,O311:5.3105565551e-11):1.88972460964e-10,(O308:5.4491612275e-11,O309:5.4491612275e-11):1.8758641424e-10):7.79355069576e-10,((O306:5.5923175022e-11,O307:5.5923175022e-11):1.99041714494e-10,(O304:5.74020592575e-11,O305:5.74020592575e-11):1.97562830258e-10):7.66468206575e-10):3.31520513233e-09,(((O302:5.89301545552e-11,O303:5.89301545552e-11):2.09790658615e-10,(O300:6.05094391044e-11,O301:6.05094391044e-11):2.08211374066e-10):8.65917616198e-10,((O298:6.21419844918e-11,O299:6.21419844918e-11):2.2127510051e-10,(O296:6.38299607755e-11,O297:6.38299607755e-11):2.19587124226e-10):8.51221344367e-10):3.20199979905e-09):1.13653456724e-08):3.62084382712e-08):2.15899391038e-07,(((((((O486:8.85263254499e-12,O487:8.85263254499e-12):3.13134049319e-11,(O484:8.99940897461e-12,O485:8.99940897461e-12):3.11666285023e-11):1.27715383041e-10,((O482:9.14923998995e-12,O483:9.14923998995e-12):3.23653953545e-11,(O480:9.30220219283e-12,O481:9.30220219283e-12):3.22124331516e-11):1.26366785174e-10):5.31161307751e-10,(((O478:9.45837443564e-12,O479:9.45837443564e-12):3.34619369177e-11,(O476:9.61783789722e-12,O477:9.61783789722e-12):3.33024734561e-11):1.36523035541e-10,((O474:9.78067616162e-12,O475:9.78067616162e-12):3.46053129776e-11,(O472:9.94697529988e-12,O473:9.94697529988e-12):3.44390138393e-11):1.35057357755e-10):5.19599381375e-10):2.29968241886e-09,((((O470:1.0116823955e-11,O471:1.0116823955e-11):3.57979445961e-11,(O468:1.02903134302e-11,O469:1.02903134302e-11):3.56244551209e-11):1.46103205358e-10,((O466:1.04675377803e-11,O467:1.04675377803e-11):3.7042400115e-11,(O464:1.06485939072e-11,O465:1.06485939072e-11):3.68613439881e-11):1.44508036014e-10):6.08451917496e-10,(((O462:1.08335816587e-11,O463:1.08335816587e-11):3.83414054848e-11,(O460:1.10226039312e-11,O461:1.10226039312e-11):3.81523832122e-11):1.56538646797e-10,((O458:1.12157667768e-11,O459:1.12157667768e-11):3.96978554154e-11,(O456:1.1413179514e-11,O457:1.1413179514e-11):3.95004426782e-11):1.54800011748e-10):5.94756257465e-10):2.19825525572e-09):1.08401455112e-08,(((((O454:1.16149548435e-11,O455:1.16149548435e-11):4.11148254229e-11,(O452:1.18212089682e-11,O453:1.18212089682e-11):4.09085712982e-11):1.67922643703e-10,((O450:1.20320617182e-11,O451:1.20320617182e-11):4.25955848523e-11,(O448:1.22476366807e-11,O449:1.22476366807e-11):4.23800098898e-11):1.66024777399e-10):7.00326776456e-10,(((O446:1.24680613353e-11,O447:1.24680613353e-11):4.41436109645e-11,(O444:1.26934671946e-11,O445:1.26934671946e-11):4.39182051052e-11):1.80360568847e-10,((O442:1.2923989951e-11,O443:1.2923989951e-11):4.57626041862e-11,(O440:1.31597696286e-11,O441:1.31597696286e-11):4.55268245086e-11):1.7828564701e-10):6.84006959279e-10):3.04961142115e-09,((((O438:1.34009507428e-11,O439:1.34009507428e-11):4.74565046298e-11,(O436:1.36476824649e-11,O437:1.36476824649e-11):4.72097729077e-11):1.93971707981e-10,((O434:1.39001187949e-11,O435:1.39001187949e-11):4.9229510001e-11,(O432:1.41584187405e-11,O433:1.41584187405e-11):4.89712100554e-11):1.91699534557e-10):8.10216500195e-10,(((O430:1.44227465046e-11,O431:1.44227465046e-11):5.10860950272e-11,(O428:1.46932716797e-11,O429:1.46932716797e-11):5.08155698522e-11):2.08891399458e-10,((O426:1.49701694511e-11,O427:1.49701694511e-11):5.30310325486e-11,(O424:1.52536208092e-11,O425:1.52536208092e-11):5.27475811905e-11):2.06399038991e-10):7.90645422558e-10):2.90554495803e-09):9.86828003675e-09):6.19117975574e-08,((((((O422:1.55438127696e-11,O423:1.55438127696e-11):5.50694164338e-11,(O420:1.5840938604e-11,O421:1.5840938604e-11):5.47722905994e-11):2.25273551246e-10,((O418:1.61451980803e-11,O419:1.61451980803e-11):5.72066864965e-11,(O416:1.64567977136e-11,O417:1.64567977136e-11):5.68950868632e-11):2.22534895872e-10):9.42531207029e-10,(((O414:1.67759510273e-11,O415:1.67759510273e-11):5.94486556086e-11,(O412:1.7102878827e-11,O413:1.7102878827e-11):5.91217278089e-11):2.43293610758e-10,((O410:1.74378094852e-11,O411:1.74378094852e-11):6.18015392278e-11,(O408:1.77809792391e-11,O409:1.77809792391e-11):6.14583694739e-11):2.40278868681e-10):9.18899770084e-10):4.13175832746e-09,((((O406:1.81326325023e-11,O407:1.81326325023e-11):6.42719875806e-11,(O404:1.84930221888e-11,O405:1.84930221888e-11):6.3911597894e-11):2.63152079881e-10,((O402:1.88624100536e-11,O403:1.88624100536e-11):6.68671207683e-11,(O400:1.92410670469e-11,O401:1.92410670469e-11):6.6488463775e-11):2.59827169142e-10):1.10299358558e-09,(((O398:1.96292736855e-11,O399:1.96292736855e-11):6.95945670955e-11,(O396:2.00273204406e-11,O397:2.00273204406e-11):6.91965203404e-11):2.8507868868e-10,((O394:2.04355081439e-11,O395:2.04355081439e-11):7.24625049514e-11,(O392:2.08541484115e-11,O393:2.08541484115e-11):7.20438646838e-11):2.81404516365e-10):1.07424775608e-09):3.92162602939e-09):2.00081552986e-08,(((((O390:2.12835640886e-11,O391:2.12835640886e-11):7.54797086137e-11,(O388:2.1724089714e-11,O389:2.1724089714e-11):7.50391829883e-11):3.09337367845e-10,((O386:2.21760720068e-11,O387:2.21760720068e-11):7.86555983897e-11,(O384:2.26398703757e-11,O385:2.26398703757e-11):7.81918000208e-11):3.05268970151e-10):1.29910496933e-09,(((O382:2.31158574528e-11,O383:2.31158574528e-11):8.20002955528e-11,(O380:2.36044196523e-11,O381:2.36044196523e-11):8.15117333533e-11):3.3623219339e-10,((O378:2.41059577566e-11,O379:2.41059577566e-11):8.55246825922e-11,(O376:2.46208875295e-11,O377:2.46208875295e-11):8.50097528193e-11):3.31717706047e-10):1.26385726348e-09):5.73947861372e-09,((((O374:2.51496403601e-11,O375:2.51496403601e-11):8.92404693501e-11,(O372:2.56926639376e-11,O373:2.56926639376e-11):8.86974457727e-11):3.66114519482e-10,((O370:2.62504229584e-11,O371:2.62504229584e-11):9.3160265696e-11,(O368:2.68233998693e-11,O369:2.68233998693e-11):9.25872887851e-11):3.61093940538e-10):1.54080614388e-09,(((O366:2.74120956461e-11,O367:2.74120956461e-11):9.72976614607e-11,(O364:2.80170306115e-11,O365:2.80170306115e-11):9.66927264952e-11):3.99391569227e-10,((O362:2.8638745294e-11,O363:2.8638745294e-11):1.01667314447e-10,(O360:2.9277801329e-11,O361:2.9277801329e-11):1.01028258412e-10):3.93795266593e-10):1.49720944674e-09):5.42337345052e-09):1.793364739e-08):5.03723366021e-08):1.92059144994e-07):5.57253633652e-07):5.39362573763e-07):1.23387731939e-05,(((((((O166:6.35211714337e-10,O167:6.35211714337e-10):2.29293332746e-09,(O164:6.66478186635e-10,O165:6.66478186635e-10):2.26166685516e-09):9.71312553366e-09,((O162:6.99692307789e-10,O163:6.99692307789e-10):2.52760826732e-09,(O160:7.35001496257e-10,O161:7.35001496257e-10):2.49229907885e-09):9.41397000035e-09):4.36681381387e-08,(((O158:7.72566354179e-10,O159:7.72566354179e-10):2.79308935623e-09,(O156:8.12562032204e-10,O157:8.12562032204e-10):2.7530936782e-09):1.18666121976e-08,((O154:8.55179755364e-10,O155:8.55179755364e-10):3.09435933819e-09,(O152:9.0062853136e-10,O153:9.0062853136e-10):3.04891056219e-09):1.14827288144e-08):4.08771408062e-08):2.23528418613e-07,((((O150:9.4913706584e-10,O151:9.4913706584e-10):3.43736708675e-09,(O148:1.0009559129e-09,O149:1.0009559129e-09):3.3855482397e-09):1.46514174848e-08,((O146:1.05635989333e-09,O147:1.05635989333e-09):3.82924775049e-09,(O144:1.11565081825e-09,O145:1.11565081825e-09):3.76995682557e-09):1.41523139936e-08):6.67303800382e-08,(((O142:1.17916056144e-09,O143:1.17916056144e-09):4.27860097454e-09,(O140:1.24725453085e-09,O141:1.24725453085e-09):4.21050700513e-09):1.83032539644e-08,((O138:1.32033559762e-09,O139:1.32033559762e-09):4.79584473179e-09,(O136:1.39884855068e-09,O137:1.39884855068e-09):4.71733177873e-09):1.7644835171e-08):6.20072861753e-08):1.94069525652e-07):1.11277620867e-06,((((((O230:1.74088496586e-10,O231:1.74088496586e-10):6.23049970639e-10,(O228:1.80236758285e-10,O229:1.80236758285e-10):6.1690170894e-10):2.59722941757e-09,((O226:1.86658854352e-10,O227:1.86658854352e-10):6.68301362912e-10,(O224:1.9336955145e-10,O225:1.9336955145e-10):6.61590665814e-10):2.53940766753e-09):1.12933049241e-08,(((O222:2.00384553669e-10,O223:2.00384553669e-10):7.17736580894e-10,(O220:2.0772057115e-10,O221:2.0772057115e-10):7.10400563413e-10):2.99642415308e-09,((O218:2.15395394433e-10,O219:2.15395394433e-10):7.71828520524e-10,(O216:2.23427975022e-10,O217:2.23427975022e-10):7.63795939934e-10):2.92732137269e-09):1.07731275213e-08):5.37172609765e-08,((((O214:2.31838512791e-10,O215:2.31838512791e-10):8.31113629205e-10,(O212:2.40648550848e-10,O213:2.40648550848e-10):8.22303591148e-10):3.4753489596e-09,((O210:2.49881078596e-10,O211:2.49881078596e-10):8.96201856574e-10,(O208:2.59560643775e-10,O209:2.59560643775e-10):8.86522291395e-10):3.39221816643e-09):1.52088524594e-08,(((O206:2.69713474357e-10,O207:2.69713474357e-10):9.67788392917e-10,(O204:2.80367611283e-10,O205:2.80367611283e-10):9.57134255991e-10):4.05389991125e-09,((O202:2.91553053107e-10,O203:2.91553053107e-10):1.04666755932e-09,(O200:3.03301913754e-10,O201:3.03301913754e-10):1.03491869867e-09):3.9531811661e-09):1.44557517825e-08):4.86577802244e-08):3.13425328537e-07,(((((O198:3.15648594732e-10,O199:3.15648594732e-10):1.13374929682e-09,(O196:3.28629973269e-10,O197:3.28629973269e-10):1.12076791829e-09):4.75801417826e-09,((O194:3.4228560804e-10,O195:3.4228560804e-10):1.23007880641e-09,(O192:3.56657964336e-10,O193:3.56657964336e-10):1.21570645012e-09):4.63504765536e-09):2.09785592572e-08,(((O190:3.71792660715e-10,O191:3.71792660715e-10):1.33686002317e-09,(O188:3.87738739466e-10,O189:3.87738739466e-10):1.32091394442e-09):5.62186033487e-09,((O186:4.04548963441e-10,O187:4.04548963441e-10):1.45548377465e-09,(O184:4.22280142167e-10,O185:4.22280142167e-10):1.43775259592e-09):5.47048028067e-09):1.98554583083e-08):1.02851991929e-07,((((O182:4.40993490456e-10,O183:4.40993490456e-10):1.58756168547e-09,(O180:4.60755023185e-10,O181:4.60755023185e-10):1.56780015274e-09):6.69096398814e-09,((O178:4.81635990307e-10,O179:4.81635990307e-10):1.73496716118e-09,(O176:5.03713356732e-10,O177:5.03713356732e-10):1.71288979476e-09):6.50291601258e-09):2.97633597046e-08,(((O174:5.27070332265e-10,O175:5.27070332265e-10):1.89988513236e-09,(O172:5.51796957472e-10,O173:5.51796957472e-10):1.87515850715e-09):8.02673434948e-09,((O170:5.77990752111e-10,O171:5.77990752111e-10):2.08487269001e-09,(O168:6.05757433643e-10,O169:6.05757433643e-10):2.05710600848e-09):7.79082637197e-09):2.80291890546e-08):9.15550843877e-08):2.51792299066e-07):1.01078377367e-06):5.67073720523e-06,(((((O102:4.35741452077e-09,O103:4.35741452077e-09):1.60347772871e-08,(O100:4.7111497953e-09,O101:4.7111497953e-09):1.56810420126e-08):7.04558051207e-08,((O98:5.10152025304e-09,O99:5.10152025304e-09):1.88112066559e-08,(O96:5.53317472451e-09,O97:5.53317472451e-09):1.83795521844e-08):6.69352700197e-08):3.42435719702e-07,(((O94:6.01146506617e-09,O95:6.01146506617e-09):2.22154456685e-08,(O92:6.54257033302e-09,O93:6.54257033302e-09):2.16843404016e-08):9.84110733912e-08,((O90:7.13364617891e-09,O91:7.13364617891e-09):2.64258219295e-08,(O88:7.79300531015e-09,O89:7.79300531015e-09):2.57664627982e-08):9.30785160174e-08):3.06645732505e-07):1.97558502303e-06,(((((O134:1.48328515623e-09,O135:1.48328515623e-09):5.39366829994e-09,(O132:1.57418991463e-09,O133:1.57418991463e-09):5.30276354154e-09):2.31672099444e-08,((O130:1.67216662335e-09,O131:1.67216662335e-09):6.08761623666e-09,(O128:1.77788587324e-09,O129:1.77788587324e-09):5.98189698677e-09):2.22843805406e-08):1.07242624118e-07,(((O126:1.89209362794e-09,O127:1.89209362794e-09):6.89684683226e-09,(O124:2.01562106324e-09,O125:2.01562106324e-09):6.77331939696e-09):2.9759798075e-08,((O122:2.14939587532e-09,O123:2.14939587532e-09):7.84512493369e-09,(O120:2.29445530558e-09,O121:2.29445530558e-09):7.70006550343e-09):2.85542177262e-08):9.87380489838e-08):5.86683122566e-07,((((O118:2.45196117663e-09,O119:2.45196117663e-09):8.96213244684e-09,(O116:2.62321729038e-09,O117:2.62321729038e-09):8.79087633309e-09):3.88735694082e-08,((O114:2.80968960797e-09,O115:2.80968960797e-09):1.02852135865e-08,(O112:3.01302971443e-09,O113:3.01302971443e-09):1.00818734801e-08):3.71927598372e-08):1.83745293941e-07,(((O110:3.23510217333e-09,O111:3.23510217333e-09):1.18617211977e-08,(O108:3.4780165011e-09,O109:3.4780165011e-09):1.161880687e-08):5.17593599239e-08,((O106:3.74416464452e-09,O107:3.74416464452e-09):1.37522032237e-08,(O104:4.03626503408e-09,O105:4.03626503408e-09):1.34601028342e-08):4.93598154267e-08):1.67176773678e-07):4.89936953112e-07):1.68489882957e-06):4.65448250157e-06):6.63984797331e-06):0.000118191594625,((((((O86:8.53033633274e-09,O87:8.53033633274e-09):3.16826592993e-08,(O84:9.35697028788e-09,O85:9.35697028788e-09):3.08560253442e-08):1.41715378715e-07,((O82:1.0286206707e-08,O83:1.0286206707e-08):3.83141807997e-08,(O80:1.13337143275e-08,O81:1.13337143275e-08):3.72666731793e-08):1.33327986841e-07):7.16217055123e-07,(((O78:1.25180259574e-08,O79:1.25180259574e-08):4.67752487266e-08,(O76:1.38611527223e-08,O77:1.38611527223e-08):4.54321219617e-08):2.11690169995e-07,((O74:1.53893505694e-08,O75:1.53893505694e-08):5.7706416853e-08,(O72:1.7134082143e-08,O73:1.7134082143e-08):5.59616852794e-08):1.97887677256e-07):6.27161984792e-07):4.8550333828e-06,((((O70:1.91332309629e-08,O71:1.91332309629e-08):7.20258488574e-08,(O68:2.14326436309e-08,O69:2.14326436309e-08):6.97264361894e-08):3.30716509879e-07,((O66:2.4088101557e-08,O67:2.4088101557e-08):9.10757454226e-08,(O64:2.71678593357e-08,O65:2.71678593357e-08):8.79959876439e-08):3.06711742719e-07):1.77257659724e-06,(((O62:3.07559366339e-08,O63:3.07559366339e-08):1.16861685041e-07,(O60:3.49564205297e-08,O61:3.49564205297e-08):1.12661201145e-07):5.46503139695e-07,((O58:3.98991349868e-08,O59:3.98991349868e-08):1.52455262265e-07,(O56:4.57471776736e-08,O57:4.57471776736e-08):1.46607219578e-07):5.01766364118e-07):1.50033142557e-06):3.55872662533e-06):4.33783418489e-05,((((O38:2.05456936226e-07,O39:2.05456936226e-07):8.21598886176e-07,(O36:2.52929428643e-07,O37:2.52929428643e-07):7.7412639376e-07):4.26166683142e-06,((O34:3.14940791131e-07,O35:3.14940791131e-07):1.27898708053e-06,(O32:3.9717718233e-07,O33:3.9717718233e-07):1.19675068933e-06):3.69479478216e-06):2.0646329729e-05,((((O54:5.27070332265e-08,O55:5.27070332265e-08):2.02693765227e-07,(O52:6.10422798364e-08,O53:6.10422798364e-08):1.94358518617e-07):9.70853129412e-07,((O50:7.10923654762e-08,O51:7.10923654762e-08):2.75441634681e-07,(O48:8.32986255727e-08,O49:8.32986255727e-08):2.63235374584e-07):8.79719927709e-07):5.74267190574e-06,(((O46:9.82407826568e-08,O47:9.82407826568e-08):3.83962780999e-07,(O44:1.16688837546e-07,O45:1.16688837546e-07):3.6551472611e-07):1.90004508738e-06,((O42:1.39677913905e-07,O43:1.39677913905e-07):5.51605392122e-07,(O40:1.68617913158e-07,O41:1.68617913158e-07):5.22665392869e-07):1.69096534501e-06):4.58667718257e-06):1.89661265492e-05):2.31964682784e-05):8.27632731781e-05):0.000944148514928,((((O22:1.64093677799e-06,O23:1.64093677799e-06):7.21289826944e-06,(O20:2.34253481007e-06,O21:2.34253481007e-06):6.51130023736e-06):4.03657861019e-05,(((O30:5.08097034339e-07,O31:5.08097034339e-07):2.10396208635e-06,(O28:6.60589245607e-07,O29:6.60589245607e-07):1.95146987509e-06):1.17515051062e-05,((O26:8.7483553092e-07,O27:8.7483553092e-07):3.71555177212e-06,(O24:1.18343195266e-06,O25:1.18343195266e-06):3.40695535038e-06):9.77317692382e-06):3.48560569225e-05):0.000276600615545,(((O18:3.46260387812e-06,O19:3.46260387812e-06):1.59884909431e-05,(O16:5.33982656243e-06,O17:5.33982656243e-06):1.41112682588e-05):0.000114847479781,((O14:8.68055555556e-06,O15:8.68055555556e-06):4.32712224436e-05,(O12:1.50947953146e-05,O13:1.50947953146e-05):3.68569826846e-05):8.23467966028e-05):0.000191521662092):0.000750223072073):0.00966394187379,(((O6:0.00015943877551,O7:0.00015943877551):0.000640713202943,((O10:2.86960514233e-05,O11:2.86960514233e-05):0.000163822569126,(O8:6.17283950617e-05,O9:6.17283950617e-05):0.000130790225487):0.000607633357904):0.00466660946978,(O3:0.00236111111111,(O4:0.000555555555556,O5:0.000555555555556):0.00180555555556):0.00310565033712):0.00527322373433):0.0773174861005,(O1:0.0138888888889,O2:0.0138888888889):0.0741685823942):0.405772855466); diff --git a/gneiss/tests/data/small_tree.nwk b/gneiss/tests/data/small_tree.nwk new file mode 100644 index 0000000..40c794f --- /dev/null +++ b/gneiss/tests/data/small_tree.nwk @@ -0,0 +1 @@ +(O0:0.316212845735,(O1:0.0544249673249,((((O8:6.17283950617e-05,O9:6.17283950617e-05):0.000396077412446,(O6:0.00015943877551,O7:0.00015943877551):0.000298367031998):0.00183128228143,(O4:0.000555555555556,O5:0.000555555555556):0.00173353253338):0.0105194882737,(O2:0.00347222222222,O3:0.00347222222222):0.00933635414042):0.0416163909622):0.26178787841); diff --git a/gneiss/tests/test_balances.py b/gneiss/tests/test_balances.py index 9a8ff72..c46bb01 100644 --- a/gneiss/tests/test_balances.py +++ b/gneiss/tests/test_balances.py @@ -9,7 +9,8 @@ from gneiss.layouts import default_layout from skbio import TreeNode from skbio.util import get_data_path - +from skbio.stats.composition import _check_orthogonality +from gneiss.util import rename_internal_nodes class TestPlot(unittest.TestCase): @@ -151,15 +152,30 @@ def test_balance_basis_large1(self): get_data_path('large_tree_basis.txt', subfolder='data')) res_basis, res_keys = balance_basis(t) + _check_orthogonality(res_basis) exp_basis = exp_basis[:, ::-1] print(exp_basis.shape) for i in range(len(res_basis)): print(i) + print(exp_basis[i]- res_basis[i]) npt.assert_allclose(exp_basis[i], res_basis[i]) - npt.assert_allclose(exp_basis[:, ::-1], res_basis) + npt.assert_allclose(exp_basis[:, ::-1], res_basis, rtol=1e-5, atol=1e-5) + + def test_balance_basis_large2(self): + fname = get_data_path('large_tree2.nwk', + subfolder='data') + t = TreeNode.read(fname) + res_basis, res_keys = balance_basis(t) + _check_orthogonality(res_basis) + def test_balance_basis_small1(self): + fname = get_data_path('small_tree.nwk', + subfolder='data') + t = TreeNode.read(fname) + res_basis, res_keys = balance_basis(t) + _check_orthogonality(res_basis) if __name__ == "__main__": unittest.main() diff --git a/gneiss/tests/test_sort.py b/gneiss/tests/test_sort.py new file mode 100644 index 0000000..1288b75 --- /dev/null +++ b/gneiss/tests/test_sort.py @@ -0,0 +1,178 @@ +# ---------------------------------------------------------------------------- +# Copyright (c) 2016--, gneiss development team. +# +# Distributed under the terms of the GPLv3 License. +# +# The full license is in the file COPYING.txt, distributed with this software. +# ---------------------------------------------------------------------------- +import numpy as np +import pandas as pd +import unittest +from gneiss.sort import niche_sort, mean_niche_estimator +import pandas.util.testing as pdt + + +class TestSort(unittest.TestCase): + def setUp(self): + pass + + def test_mean_niche_estimator1(self): + gradient = pd.Series( + [1, 2, 3, 4, 5], + index=['s1', 's2', 's3', 's4', 's5']) + values = pd.Series( + [1, 1, 0, 0, 0], + index=['s1', 's2', 's3', 's4', 's5']) + m = mean_niche_estimator(values, gradient) + self.assertEqual(m, 1.5) + + def test_mean_niche_estimator2(self): + gradient = pd.Series( + [1, 2, 3, 4, 5], + index=['s1', 's2', 's3', 's4', 's5']) + values = pd.Series( + [1, 3, 0, 0, 0], + index=['s1', 's2', 's3', 's4', 's5']) + m = mean_niche_estimator(values, gradient) + self.assertEqual(m, 1.75) + + def test_mean_niche_estimator_bad_length(self): + gradient = pd.Series( + [1, 2, 3, 4, 5], + index=['s1', 's2', 's3', 's4', 's5']) + values = pd.Series( + [1, 3, 0, 0, 0, 0], + index=['s1', 's2', 's3', 's4', 's5', 's6']) + + with self.assertRaises(ValueError): + mean_niche_estimator(values, gradient) + + def test_mean_niche_estimator_missing(self): + gradient = pd.Series( + [1, 2, 3, 4, np.nan], + index=['s1', 's2', 's3', 's4', 's5']) + values = pd.Series( + [1, 3, 0, 0, 0], + index=['s1', 's2', 's3', 's4', 's5']) + + with self.assertRaises(ValueError): + mean_niche_estimator(values, gradient) + + def test_basic_niche_sort(self): + table = pd.DataFrame( + [[1, 1, 0, 0, 0], + [0, 1, 1, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s1', 's2', 's3', 's4', 's5'], + index=['o1', 'o2', 'o3', 'o4']).T + gradient = pd.Series( + [1, 2, 3, 4, 5], + index=['s1', 's2', 's3', 's4', 's5']) + res_table = niche_sort(table, gradient) + pdt.assert_frame_equal(table, res_table) + + def test_basic_niche_sort_error(self): + table = pd.DataFrame( + [[1, 1, 0, 0, 0], + [0, 1, 1, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s1', 's2', 's3', 's4', 's5'], + index=['o1', 'o2', 'o3', 'o4']).T + gradient = pd.Series( + [1, 2, 3, 4, 5], + index=['s1', 's2', 's3', 's4', 's5']) + with self.assertRaises(ValueError): + niche_sort(table, gradient, niche_estimator='rawr') + + def test_basic_niche_sort_scrambled(self): + # Swap samples s1 and s2 and features o1 and o2 to see if this can + # obtain the original table structure. + table = pd.DataFrame( + [[1, 0, 1, 0, 0], + [1, 1, 0, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s2', 's1', 's3', 's4', 's5'], + index=['o2', 'o1', 'o3', 'o4']).T + + gradient = pd.Series( + [2, 1, 3, 4, 5], + index=['s2', 's1', 's3', 's4', 's5']) + + exp_table = pd.DataFrame( + [[1, 1, 0, 0, 0], + [0, 1, 1, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s1', 's2', 's3', 's4', 's5'], + index=['o1', 'o2', 'o3', 'o4']).T + + res_table = niche_sort(table, gradient) + + pdt.assert_frame_equal(exp_table, res_table) + + def test_basic_niche_sort_lambda(self): + table = pd.DataFrame( + [[1, 1, 0, 0, 0], + [0, 0, 1, 1, 0], + [0, 1, 1, 0, 0], + [0, 0, 0, 1, 1]], + columns=['s1', 's2', 's3', 's4', 's5'], + index=['o1', 'o3', 'o2', 'o4']).T + gradient = pd.Series( + [1, 2, 3, 4, 5], + index=['s1', 's2', 's3', 's4', 's5']) + + exp_table = pd.DataFrame( + [[1, 1, 0, 0, 0], + [0, 1, 1, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s1', 's2', 's3', 's4', 's5'], + index=['o1', 'o2', 'o3', 'o4']).T + + def _dumb_estimator(v, gradient): + v[v > 0] = 1 + values = v / v.sum() + return np.dot(gradient, values) + + res_table = niche_sort(table, gradient, + niche_estimator=_dumb_estimator) + pdt.assert_frame_equal(exp_table, res_table) + + def test_basic_niche_sort_immutable(self): + # Swap samples s1 and s2 and features o1 and o2 to see if this can + # obtain the original table structure. + table = pd.DataFrame( + [[1, 0, 1, 0, 0], + [1, 1, 0, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s2', 's1', 's3', 's4', 's5'], + index=['o2', 'o1', 'o3', 'o4']).T + + gradient = pd.Series( + [2, 1, 3, 4, 5], + index=['s2', 's1', 's3', 's4', 's5']) + + exp_table = pd.DataFrame( + [[1, 0, 1, 0, 0], + [1, 1, 0, 0, 0], + [0, 0, 1, 1, 0], + [0, 0, 0, 1, 1]], + columns=['s2', 's1', 's3', 's4', 's5'], + index=['o2', 'o1', 'o3', 'o4']).T + + exp_gradient = pd.Series( + [2, 1, 3, 4, 5], + index=['s2', 's1', 's3', 's4', 's5']) + + niche_sort(table, gradient) + pdt.assert_frame_equal(exp_table, table) + pdt.assert_series_equal(exp_gradient, gradient) + + +if __name__ == '__main__': + unittest.main() diff --git a/gneiss/tests/test_util.py b/gneiss/tests/test_util.py new file mode 100644 index 0000000..fecf40f --- /dev/null +++ b/gneiss/tests/test_util.py @@ -0,0 +1,313 @@ +# ---------------------------------------------------------------------------- +# Copyright (c) 2016--, gneiss development team. +# +# Distributed under the terms of the GPLv3 License. +# +# The full license is in the file COPYING.txt, distributed with this software. +# ---------------------------------------------------------------------------- + +import unittest +import pandas as pd +import pandas.util.testing as pdt +from skbio import TreeNode +from gneiss.util import match, match_tips, rename_internal_nodes + + +class TestUtil(unittest.TestCase): + + def test_match(self): + table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + metadata = pd.DataFrame([['a', 'control'], + ['b', 'control'], + ['c', 'diseased'], + ['d', 'diseased']], + index=['s1', 's2', 's3', 's4'], + columns=['Barcode', 'Treatment']) + exp_table, exp_metadata = table, metadata + res_table, res_metadata = match(table, metadata) + + # make sure that the metadata and table indeces match + pdt.assert_index_equal(res_table.index, res_metadata.index) + + res_table = res_table.sort_index() + exp_table = exp_table.sort_index() + + res_metadata = res_metadata.sort_index() + exp_metadata = exp_metadata.sort_index() + + pdt.assert_frame_equal(exp_table, res_table) + pdt.assert_frame_equal(exp_metadata, res_metadata) + + def test_match_immutable(self): + # tests to make sure that the original tables don't change. + table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + metadata = pd.DataFrame([['a', 'control'], + ['c', 'diseased'], + ['b', 'control']], + index=['s1', 's3', 's2'], + columns=['Barcode', 'Treatment']) + + exp_table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + exp_metadata = pd.DataFrame([['a', 'control'], + ['c', 'diseased'], + ['b', 'control']], + index=['s1', 's3', 's2'], + columns=['Barcode', 'Treatment']) + match(table, metadata) + pdt.assert_frame_equal(table, exp_table) + pdt.assert_frame_equal(metadata, exp_metadata) + + def test_match_duplicate(self): + table1 = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s2', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + metadata1 = pd.DataFrame([['a', 'control'], + ['b', 'control'], + ['c', 'diseased'], + ['d', 'diseased']], + index=['s1', 's2', 's3', 's4'], + columns=['Barcode', 'Treatment']) + + table2 = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + metadata2 = pd.DataFrame([['a', 'control'], + ['b', 'control'], + ['c', 'diseased'], + ['d', 'diseased']], + index=['s1', 's1', 's3', 's4'], + columns=['Barcode', 'Treatment']) + + with self.assertRaises(ValueError): + match(table1, metadata1) + with self.assertRaises(ValueError): + match(table2, metadata2) + + def test_match_scrambled(self): + table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + metadata = pd.DataFrame([['a', 'control'], + ['c', 'diseased'], + ['b', 'control'], + ['d', 'diseased']], + index=['s1', 's3', 's2', 's4'], + columns=['Barcode', 'Treatment']) + exp_table = table + exp_metadata = pd.DataFrame([['a', 'control'], + ['b', 'control'], + ['c', 'diseased'], + ['d', 'diseased']], + index=['s1', 's2', 's3', 's4'], + columns=['Barcode', 'Treatment']) + + res_table, res_metadata = match(table, metadata) + # make sure that the metadata and table indeces match + pdt.assert_index_equal(res_table.index, res_metadata.index) + + res_table = res_table.sort_index() + exp_table = exp_table.sort_index() + + res_metadata = res_metadata.sort_index() + exp_metadata = exp_metadata.sort_index() + + pdt.assert_frame_equal(exp_table, res_table) + pdt.assert_frame_equal(exp_metadata, res_metadata) + + def test_match_intersect(self): + table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['o1', 'o2', 'o3', 'o4']) + metadata = pd.DataFrame([['a', 'control'], + ['c', 'diseased'], + ['b', 'control']], + index=['s1', 's3', 's2'], + columns=['Barcode', 'Treatment']) + + exp_table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3]], + index=['s1', 's2', 's3'], + columns=['o1', 'o2', 'o3', 'o4']) + + exp_metadata = pd.DataFrame([['a', 'control'], + ['b', 'control'], + ['c', 'diseased']], + index=['s1', 's2', 's3'], + columns=['Barcode', 'Treatment']) + + res_table, res_metadata = match(table, metadata) + # sort for comparison, since the match function + # scrambles the names due to hashing. + res_table = res_table.sort_index() + res_metadata = res_metadata.sort_index() + pdt.assert_frame_equal(exp_table, res_table) + pdt.assert_frame_equal(exp_metadata, res_metadata) + + def test_match_tips(self): + table = pd.DataFrame([[0, 0, 1, 1], + [2, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'c', 'd']) + tree = TreeNode.read([u"(((a,b)f, c),d)r;"]) + exp_table, exp_tree = table, tree + res_table, res_tree = match_tips(table, tree) + pdt.assert_frame_equal(exp_table, res_table) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_match_tips_scrambled_tips(self): + table = pd.DataFrame([[0, 0, 1, 1], + [2, 3, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'c', 'd']) + tree = TreeNode.read([u"(((b,a)f, c),d)r;"]) + exp_tree = tree + exp_table = pd.DataFrame([[0, 0, 1, 1], + [3, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['b', 'a', 'c', 'd']) + + res_table, res_tree = match_tips(table, tree) + pdt.assert_frame_equal(exp_table, res_table) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_match_tips_scrambled_columns(self): + table = pd.DataFrame([[0, 0, 1, 1], + [3, 2, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['b', 'a', 'c', 'd']) + tree = TreeNode.read([u"(((a,b)f, c),d)r;"]) + exp_tree = tree + exp_table = pd.DataFrame([[0, 0, 1, 1], + [2, 3, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'c', 'd']) + + res_table, res_tree = match_tips(table, tree) + pdt.assert_frame_equal(exp_table, res_table) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_match_tips_intersect_tips(self): + # there are less tree tips than table columns + table = pd.DataFrame([[0, 0, 1, 1], + [2, 3, 4, 4], + [5, 5, 3, 3], + [0, 0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'c', 'd']) + tree = TreeNode.read([u"((a,b)f,d)r;"]) + exp_table = pd.DataFrame([[0, 0, 1], + [2, 3, 4], + [5, 5, 3], + [0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'd']) + exp_tree = tree + res_table, res_tree = match_tips(table, tree) + pdt.assert_frame_equal(exp_table, res_table) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_match_tips_intersect_columns(self): + # table has less columns than tree tips + table = pd.DataFrame([[0, 0, 1], + [2, 3, 4], + [5, 5, 3], + [0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'd']) + tree = TreeNode.read([u"(((a,b)f, c),d)r;"]) + exp_table = pd.DataFrame([[1, 0, 0], + [4, 2, 3], + [3, 5, 5], + [1, 0, 0]], + index=['s1', 's2', 's3', 's4'], + columns=['d', 'a', 'b']) + exp_tree = TreeNode.read([u"(d,(a,b)f)r;"]) + res_table, res_tree = match_tips(table, tree) + pdt.assert_frame_equal(exp_table, res_table) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_match_tips_intersect_tree_immutable(self): + # tests to see if tree chnages. + table = pd.DataFrame([[0, 0, 1], + [2, 3, 4], + [5, 5, 3], + [0, 0, 1]], + index=['s1', 's2', 's3', 's4'], + columns=['a', 'b', 'd']) + tree = TreeNode.read([u"(((a,b)f, c),d)r;"]) + match_tips(table, tree) + self.assertEqual(str(tree), u"(((a,b)f,c),d)r;\n") + + def test_rename_internal_nodes(self): + tree = TreeNode.read([u"(((a,b), c),d)r;"]) + exp_tree = TreeNode.read([u"(((a,b)y2, c)y1,d)y0;"]) + res_tree = rename_internal_nodes(tree) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_rename_internal_nodes_names(self): + tree = TreeNode.read([u"(((a,b), c),d)r;"]) + exp_tree = TreeNode.read([u"(((a,b)ab, c)abc,d)r;"]) + res_tree = rename_internal_nodes(tree, ['r', 'abc', 'ab']) + self.assertEqual(str(exp_tree), str(res_tree)) + + def test_rename_internal_nodes_names_mismatch(self): + tree = TreeNode.read([u"(((a,b), c),d)r;"]) + with self.assertRaises(ValueError): + rename_internal_nodes(tree, ['r', 'abc']) + + def test_rename_internal_nodes_warning(self): + tree = TreeNode.read([u"(((a,b)y2, c),d)r;"]) + with self.assertWarns(Warning): + rename_internal_nodes(tree) + + def test_rename_internal_nodes_immutable(self): + tree = TreeNode.read([u"(((a,b)y2, c),d)r;"]) + rename_internal_nodes(tree) + self.assertEqual(str(tree), "(((a,b)y2,c),d)r;\n") + + def test_rename_internal_nodes_mutable(self): + tree = TreeNode.read([u"(((a,b)y2, c),d)r;"]) + rename_internal_nodes(tree, inplace=True) + self.assertEqual(str(tree), "(((a,b)y2,c)y1,d)y0;\n") + + +if __name__ == '__main__': + unittest.main() diff --git a/gneiss/util.py b/gneiss/util.py new file mode 100644 index 0000000..98e7031 --- /dev/null +++ b/gneiss/util.py @@ -0,0 +1,162 @@ +# ---------------------------------------------------------------------------- +# Copyright (c) 2016--, gneiss development team. +# +# Distributed under the terms of the GPLv3 License. +# +# The full license is in the file COPYING.txt, distributed with this software. +# ---------------------------------------------------------------------------- +import warnings + + +def match(table, metadata): + """ Matches samples between a contingency table and a metadata table. + + Sorts samples in metadata and contingency table in the same order. + If there are sames contained in the contigency table, but not in metadata + or vice versa, the intersection of samples in the contingency table and the + metadata table will returned. + + Parameters + ---------- + table : pd.DataFrame + Contingency table where samples correspond to rows and + features correspond to columns. + metadata: pd.DataFrame + Metadata table where samples correspond to rows and + explanatory metadata variables correspond to columns. + + Returns + ------- + pd.DataFrame : + Filtered contingency table. + pd.DataFrame : + Filtered metadata table + + Raises + ------ + ValueError: + Raised if duplicate sample ids are present in `table`. + ValueError: + Raised if duplicate sample ids are present in `metadata`. + ValueError: + Raised if `table` and `metadata` have incompatible sizes. + + """ + subtableids = set(table.index) + submetadataids = set(metadata.index) + if len(subtableids) != len(table.index): + raise ValueError("`table` has duplicate sample ids.") + if len(submetadataids) != len(metadata.index): + raise ValueError("`metadata` has duplicate sample ids.") + + idx = subtableids & submetadataids + subtable = table.loc[idx] + submetadata = metadata.loc[idx] + return subtable, submetadata + + +def match_tips(table, tree): + """ Returns the contingency table and tree with matched tips. + + Sorts the columns of the contingency table to match the tips in + the tree. The ordering of the tips is in post-traversal order. + + If the tree is multi-furcating, then the tree is reduced to a + bifurcating tree by randomly inserting internal nodes. + + The intersection of samples in the contingency table and the + tree will returned. + + Parameters + ---------- + table : pd.DataFrame + Contingency table where samples correspond to rows and + features correspond to columns. + tree : skbio.TreeNode + Tree object where the leafs correspond to the features. + + Returns + ------- + pd.DataFrame : + Subset of the original contingency table with the common features. + skbio.TreeNode : + Sub-tree with the common features. + + Raises + ------ + ValueError: + Raised if `table` and `tree` have incompatible sizes. + + See Also + -------- + skbio.TreeNode.bifurcate + skbio.TreeNode.tips + """ + tips = [x.name for x in tree.tips()] + common_tips = list(set(tips) & set(table.columns)) + + _table = table.loc[:, common_tips] + _tree = tree.shear(names=common_tips) + + _tree.bifurcate() + _tree.prune() + sorted_features = [n.name for n in _tree.tips()] + _table = _table.reindex_axis(sorted_features, axis=1) + + return _table, _tree + + +def rename_internal_nodes(tree, names=None, inplace=False): + """ Names the internal according to level ordering. + + The tree will be traversed in level order (i.e. top-down, left to right). + If `names` is not specified, the node with the smallest label (y0) + will be located at the root of the tree, and the node with the largest + label will be located at bottom right corner of the tree. + + Parameters + ---------- + tree : skbio.TreeNode + Tree object where the leafs correspond to the features. + names : list, optional + List of labels to rename the tip names. It is assumed that the + names are listed in level ordering, and the length of the list + is at least as long as the number of internal nodes. + inplace : bool, optional + Specifies if the operation should be done on the original tree or not. + + Returns + ------- + skbio.TreeNode + Tree with renamed internal nodes. + + Raises + ------ + ValueError: + Raised if `tree` and `name` have incompatible sizes. + """ + if inplace: + _tree = tree + else: + _tree = tree.copy() + + non_tips = [n for n in _tree.levelorder() if not n.is_tip()] + if names is not None and len(non_tips) != len(names): + raise ValueError("`_tree` and `names` have incompatible sizes, " + "`_tree` has %d tips, `names` has %d elements." % + (len(non_tips), len(names))) + + i = 0 + for n in _tree.levelorder(): + if not n.is_tip(): + if names is None: + label = 'y%i' % i + else: + label = names[i] + if n.name is not None and label == n.name: + warnings.warn("Warning. Internal node (%s) has been replaced " + "with (%s)" % (n.name, label)) + + n.name = label + i += 1 + return _tree diff --git a/ipynb/balance_trees.ipynb b/ipynb/balance_trees.ipynb index 13d41be..96f22b9 100644 --- a/ipynb/balance_trees.ipynb +++ b/ipynb/balance_trees.ipynb @@ -45,7 +45,7 @@ "$$\n", "b_i = \\sqrt{\\frac{l_i r_i}{l_i + r_i}} \n", " \\log \\big( \\frac{(\\prod_{x_n \\in L}{x_n})^{1/l_i}}\n", - " {(\\prod_{x_m \\in L}{x_m})^{1/r_i}} \\big)\n", + " {(\\prod_{x_m \\in R}{x_m})^{1/r_i}} \\big)\n", "$$\n", "\n", "where $b_i$ is the $i$th balance, $l_i$ is the number of leaves in the left subtree, $r_i$ is the number of leaves in the right subtree $x_n$ are the abundances of the species in the left subtree, and $x_m$ are the abundances of species in the right subtree.\n" diff --git a/setup.py b/setup.py index 2d9f4f7..28a305e 100644 --- a/setup.py +++ b/setup.py @@ -35,13 +35,13 @@ def finalize_options(self): extensions = cythonize(extensions) classes = """ - Development Status :: 0 - pre-alpha + Development Status :: 2 - Pre-Alpha License :: OSI Approved :: BSD License Topic :: Software Development :: Libraries Topic :: Scientific/Engineering Topic :: Scientific/Engineering :: Bio-Informatics - Programming Language :: Python :: 2 - Programming Language :: Python :: 2 :: Only + Programming Language :: Python :: 3 + Programming Language :: Python :: 3 :: Only Operating System :: Unix Operating System :: POSIX Operating System :: MacOS :: MacOS X