Contributed by Steven Rand.

Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
(cherry picked from commit 5a40bafd)

(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
(cherry picked from commit 920847ed)

HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge.

Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
(cherry picked from commit c5ee7fde)

(cherry picked from commit 924def78)
(cherry picked from commit acf20c83)
(cherry picked from commit 518705b8)

HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane

(cherry picked from commit f4329990)
(cherry picked from commit a1468668)
(cherry picked from commit f0df481c)

(cherry picked from commit 71c23c9f)
(cherry picked from commit ed4388f9)
(cherry picked from commit 674b9cda)

(cherry picked from commit ccf2d662)
(cherry picked from commit d3a6124f)
(cherry picked from commit 87abe01a)

(cherry picked from commit e015b563)
(cherry picked from commit 01624f0c)
(cherry picked from commit 5eaec560)

HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane

(cherry picked from commit ef34bf2b)
(cherry picked from commit 881b6349)
(cherry picked from commit 8a51e4f8)

(cherry picked from commit 4113ec5f)
(cherry picked from commit 7fd0556b)
(cherry picked from commit f1671e95)

HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane

(cherry picked from commit edf149b9)
(cherry picked from commit b4b4ca91)

HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand

(cherry picked from commit f597f4c4)
(cherry picked from commit d2d0872f)
(cherry picked from commit 20fa7664)

(cherry picked from commit f01a69f8)
(cherry picked from commit 120f6803)
(cherry picked from commit dd5d0103)

(cherry picked from commit 5e74196e)
(cherry picked from commit 6e75c1e2)
(cherry picked from commit 3f23d50c)

This reverts commit f70b9d92.

(cherry picked from commit 67d2875f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java

(cherry picked from commit 83f1e3b1)

YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong

YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang

(cherry picked from commit 05237636)

Contributed by Steve Loughran

(cherry picked from commit 51fefbe3)

HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures.
Contributed by Steve Loughran.

(cherry picked from commit 05ab09ca)

HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by  Sean Mackrory.

(cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)

HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan

(cherry picked from commit 274c02d2)

Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit f102c4f6.

HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp.

(cherry picked from commit 22f8b661)

Revert "HADOOP-13119. Add ability to secure log servlet using proxy users.  Contribute by Yuanbo Liu."

This reverts commit e86c8321.

(cherry picked from commit a847903b)

YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer.

(cherry picked from commit b9136773)
(cherry picked from commit c5203e170ddd68fc8532792878c37d4762068472)
(cherry picked from commit 30b3232a71afbfc177493510ea97adea5cb8b418)

(cherry picked from commit ef501b1a)
(cherry picked from commit 384803d09ac45886e74a0501f4b419a2b756c20c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

(cherry picked from commit cc5965b54138aa51f5309d5683b3879f126d5d3b)

(cherry picked from commit 17634672)
(cherry picked from commit c6c29d00)
(cherry picked from commit b8b8b9a3)

(cherry picked from commit fbcc45c8)

(cherry picked from commit e459092a)

HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory.

(cherry picked from commit d51f8ba8)
(cherry picked from commit b089e122)
(cherry picked from commit b7b8671c)

YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe.

(cherry picked from commit dbac88ba)

(cherry picked from commit 603cbcd5)

YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena)

(cherry picked from commit f59e36b4)

(cherry picked from commit 94f1d06f)
(cherry picked from commit 7b5b57e5)