Branch 2.7.1#127
Closed
rajeshdb wants to merge 7841 commits intotrunkfrom branch-2.7.1
+1,288,797-400,521
Commits
This pull request is big! We're only showing the most recent 250 commits
Commits on Mar 18, 2015
- committed
- committed
Haohui Mai HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao)
committed- committedHaohui Mai
Commits on Mar 19, 2015
- committedColin Patrick Mccabe
HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao.
committedHDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao.
committedHDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe)
committedColin Patrick Mccabe- committedColin Patrick Mccabe
- committedHaohui Mai
- committed
- committed
- committed
- committed
MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for user-specified task output subdirs. Contributed by Gera Shegalov and Siqi Li
committed- committed
- committed
Commits on Mar 20, 2015
YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong
committed- committed
YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda)
committed- committed
HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B
committed
Commits on Mar 22, 2015
Commits on Mar 23, 2015
YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R.
committed- committedBrandon Li
- committed
- committed
HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu.
committed
Commits on Mar 24, 2015
- committed
HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula
committedTsz-Wo Nicholas SzeHDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu.
committedHADOOP-11609. Correct credential commands info in CommandsManual.html#credential. Contributed by Varun Saxena.
committedMAPREDUCE-6285. ClientServiceDelegate should not retry upon AuthenticationException. Contributed by Jonathan Eagles.
committed- committed
HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li
committedBrandon Li- committedBrandon Li
HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh.
authored andJitendra PandeycommittedJitendra Pandey
Commits on Mar 25, 2015
- committed
- committed
MAPREDUCE-6292. Use org.junit package instead of junit.framework in TestCombineFileInputFormat. (aajisaka)
committed- committed
HADOOP-10670. Allow AuthenticationFilters to load secret from signature secret files. Contributed by Kai Zheng.
committedHaohui Mai- committedHaohui Mai
Commits on Mar 26, 2015
Commits on Mar 27, 2015
HADOOP-11748. The secrets of auth cookies should not be specified in configuration in clear text. Contributed by Li Lu and Haohui Mai.
committedHaohui MaiHADOOP-11257. Update "hadoop jar" documentation to warn against using it for launching yarn jars, addendum to print warning to stderr. Contributed by Masatake Iwasaki.
committed- committed
HADOOP-11639. Clean up Windows native code compilation warnings related to Windows Secure Container Executor. Contributed by Remus Rusanu.
committed
Commits on Mar 30, 2015
- committed
YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla.
committedMAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter.
committedHADOOP-11761. Fix findbugs warnings in org.apache.hadoop.security.authentication. Contributed by Li Lu.
committedHaohui MaiHADOOP-11754. RM fails to start in non-secure mode due to authentication filter failure. Contributed by Haohui Mai.
committedHaohui MaiHDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai.
committedHaohui MaiRevert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter."
committed
Commits on Mar 31, 2015
Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu."
committedRevert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)"
committedRevert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)"
committed
Commits on Apr 1, 2015
YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv)
committedHADOOP-11787. OpensslSecureRandom.c pthread_threadid_np usage signature is wrong on 32-bit Mac. Contributed by Kiran Kumar M R.
committed- committed
Revert "YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong."
committedHDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao.
committedHaohui Mai
Commits on Apr 2, 2015
HADOOP-11757. NFS gateway should shutdown when it can't start UDP or TCP server. Contributed by Brandon Li
committedBrandon Li- committedBrandon Li
HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao
committedTsz-Wo Nicholas SzeMAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe.
committed
Commits on Apr 3, 2015
HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao.
committedHDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao.
committedHDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe)
committedColin Patrick MccabeHDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe)
committedColin Patrick Mccabe
Commits on Apr 4, 2015
Commits on Apr 6, 2015
Commits on Apr 7, 2015
HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko.
committedHDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao.
committedHADOOP-11796. Skip TestShellBasedIdMapping.testStaticMapUpdate on Windows. Contributed by Xiaoyu Yao.
committed
Commits on Apr 8, 2015
- committed
HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal)
committed- committedHaohui Mai
- committedHaohui Mai
Commits on Apr 9, 2015
HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma.
committed- committedHaohui Mai
- committedColin Patrick Mccabe
YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp.
committed- committed
- committed
Commits on Apr 10, 2015
Commits on Apr 11, 2015
Commits on Apr 14, 2015
Commits on Apr 15, 2015
YARN-3462. Patches applied for YARN-2424 are inconsistent between trunk and branch-2. Contributed by Naganarasimha G R.
committedHarsh JHDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao.
committedHDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina.
committed- committed
Commits on Apr 17, 2015
Commits on Apr 18, 2015
Commits on Apr 21, 2015
Commits on Apr 22, 2015
Commits on Apr 23, 2015
- committed
YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen
committedHDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore
committedTsz-Wo Nicholas SzeHADOOP-11872. "hadoop dfs" command prints message about using "yarn jar" on Windows(branch-2 only). Contributed by Varun Vasudev.
committed- committed
Commits on Apr 24, 2015
YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal
MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter)
HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe)
committedColin Patrick Mccabe- committedColin Patrick Mccabe
- committedColin Patrick Mccabe
Commits on Apr 27, 2015
Commits on Apr 28, 2015
- committedDevaraj K
HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su
committedTsz-Wo Nicholas SzeMAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne
committed
Commits on Apr 29, 2015
HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai.
committedHaohui MaiYARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha)
committedHDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai.
committedHaohui Mai
Commits on Apr 30, 2015
- committedDevaraj K
HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao.
committed- committed
YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan.
- committed
Commits on May 1, 2015
HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh)
committedHDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe)
committedColin Patrick MccabeYARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda)
committed
Commits on May 5, 2015
Commits on May 6, 2015
Commits on May 7, 2015
Commits on May 8, 2015
- committedDevaraj K
- committed
- committed
YARN-3554. Default value for maximum nodemanager connect wait time is too high. Contributed by Naganarasimha G R
committedHDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula.
committedHDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
committedYARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith
committed
Commits on May 11, 2015
Commits on May 12, 2015
backport YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed (Jian He) to branch-2.7
committedMAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch.
committed
Commits on May 13, 2015
YARN-3539. Updated timeline server documentation and marked REST APIs evolving. Contributed by Steve Loughran.
committed- committed
HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore
committedTsz-Wo Nicholas SzeHADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth.
committedYARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch
committedYARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt.
committedYARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula
committedYARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when exceptions happen in stopping NM's sub-services. Contributed by Junping Du
committed
Commits on May 15, 2015
Commits on May 16, 2015
Commits on May 18, 2015
- committedTsz-Wo Nicholas Sze
Commits on May 19, 2015
- committed
HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts.
committedHADOOP-11973. Ensure ZkDelegationTokenSecretManager namespace znodes get created with ACLs. (Gregory Chanan via asuresh)
committedMAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du.
Commits on May 20, 2015
YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli.
committedBackport "YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan" to 2.7.1
committedYARN-3681. yarn cmd says "could not find main class 'queue'" in windows. Contributed by Craig Welch and Varun Saxena
committed
Commits on May 21, 2015
- committed
- committed
- committedDevaraj K
HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran.
committedYARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha)
committed
Commits on May 22, 2015
Commits on Jun 2, 2015
- committed
YARN-3753. RM failed to come up with "java.io.IOException: Wait for ZKClient creation timed out". Contributed by Jian He
committed- committedColin Patrick Mccabe
Commits on Jun 3, 2015
- committedColin Patrick Mccabe
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
committedHDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina)
committedYARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks
committed
Commits on Jun 4, 2015
Commits on Jun 5, 2015
Commits on Jun 6, 2015
Commits on Jun 10, 2015
Commits on Jun 11, 2015
Commits on Jun 12, 2015
Commits on Jun 14, 2015
HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang.
committedYongjun ZhangHDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal)
committed
Commits on Jun 15, 2015
Commits on Jun 16, 2015
- committedTsz-Wo Nicholas Sze
YARN-3711. Documentation of ResourceManager HA should explain configurations about listen addresses. Contributed by Masatake Iwasaki.
committedHDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula)
committedYongjun Zhang- committed
- committed
Commits on Jun 17, 2015
Commits on Jun 18, 2015
Commits on Jun 19, 2015
Commits on Jun 20, 2015
Commits on Jun 22, 2015
Commits on Jun 23, 2015
Commits on Jun 28, 2015
Commits on Jun 29, 2015
Commits on Jul 6, 2015
Commits on Jul 24, 2015
YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino)
authored andcarlo curinocommittedcarlo curinoRevert "YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino)" moving this to branch-2.7 per Akira's request.
authored andcarlo curinocommittedcarlo curino