hadoop-tools (empty) → 0.2

raw patch · 4 files changed

+685/−0 lines, 4 filesdep +basedep +boxesdep +bytestringsetup-changed

Dependencies added: base, boxes, bytestring, cereal, cereal-conduit, conduit, conduit-extra, configurator, directory, filepath, network, network-simple, old-locale, optparse-applicative, protobuf, socks, split, text, time, transformers, unix, unordered-containers, xmlhtml

Files

• LICENSE +202/−0
• Setup.hs +2/−0
• hadoop-tools.cabal +45/−0
• src/Main.hs +436/−0

@@ -0,0 +1,202 @@++                                 Apache License+                           Version 2.0, January 2004+                        http://www.apache.org/licenses/++   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++   1. Definitions.++      "License" shall mean the terms and conditions for use, reproduction,+      and distribution as defined by Sections 1 through 9 of this document.++      "Licensor" shall mean the copyright owner or entity authorized by+      the copyright owner that is granting the License.++      "Legal Entity" shall mean the union of the acting entity and all+      other entities that control, are controlled by, or are under common+      control with that entity. For the purposes of this definition,+      "control" means (i) the power, direct or indirect, to cause the+      direction or management of such entity, whether by contract or+      otherwise, or (ii) ownership of fifty percent (50%) or more of the+      outstanding shares, or (iii) beneficial ownership of such entity.++      "You" (or "Your") shall mean an individual or Legal Entity+      exercising permissions granted by this License.++      "Source" form shall mean the preferred form for making modifications,+      including but not limited to software source code, documentation+      source, and configuration files.++      "Object" form shall mean any form resulting from mechanical+      transformation or translation of a Source form, including but+      not limited to compiled object code, generated documentation,+      and conversions to other media types.++      "Work" shall mean the work of authorship, whether in Source or+      Object form, made available under the License, as indicated by a+      copyright notice that is included in or attached to the work+      (an example is provided in the Appendix below).++      "Derivative Works" shall mean any work, whether in Source or Object+      form, that is based on (or derived from) the Work and for which the+      editorial revisions, annotations, elaborations, or other modifications+      represent, as a whole, an original work of authorship. For the purposes+      of this License, Derivative Works shall not include works that remain+      separable from, or merely link (or bind by name) to the interfaces of,+      the Work and Derivative Works thereof.++      "Contribution" shall mean any work of authorship, including+      the original version of the Work and any modifications or additions+      to that Work or Derivative Works thereof, that is intentionally+      submitted to Licensor for inclusion in the Work by the copyright owner+      or by an individual or Legal Entity authorized to submit on behalf of+      the copyright owner. For the purposes of this definition, "submitted"+      means any form of electronic, verbal, or written communication sent+      to the Licensor or its representatives, including but not limited to+      communication on electronic mailing lists, source code control systems,+      and issue tracking systems that are managed by, or on behalf of, the+      Licensor for the purpose of discussing and improving the Work, but+      excluding communication that is conspicuously marked or otherwise+      designated in writing by the copyright owner as "Not a Contribution."++      "Contributor" shall mean Licensor and any individual or Legal Entity+      on behalf of whom a Contribution has been received by Licensor and+      subsequently incorporated within the Work.++   2. Grant of Copyright License. Subject to the terms and conditions of+      this License, each Contributor hereby grants to You a perpetual,+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable+      copyright license to reproduce, prepare Derivative Works of,+      publicly display, publicly perform, sublicense, and distribute the+      Work and such Derivative Works in Source or Object form.++   3. Grant of Patent License. Subject to the terms and conditions of+      this License, each Contributor hereby grants to You a perpetual,+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable+      (except as stated in this section) patent license to make, have made,+      use, offer to sell, sell, import, and otherwise transfer the Work,+      where such license applies only to those patent claims licensable+      by such Contributor that are necessarily infringed by their+      Contribution(s) alone or by combination of their Contribution(s)+      with the Work to which such Contribution(s) was submitted. If You+      institute patent litigation against any entity (including a+      cross-claim or counterclaim in a lawsuit) alleging that the Work+      or a Contribution incorporated within the Work constitutes direct+      or contributory patent infringement, then any patent licenses+      granted to You under this License for that Work shall terminate+      as of the date such litigation is filed.++   4. Redistribution. You may reproduce and distribute copies of the+      Work or Derivative Works thereof in any medium, with or without+      modifications, and in Source or Object form, provided that You+      meet the following conditions:++      (a) You must give any other recipients of the Work or+          Derivative Works a copy of this License; and++      (b) You must cause any modified files to carry prominent notices+          stating that You changed the files; and++      (c) You must retain, in the Source form of any Derivative Works+          that You distribute, all copyright, patent, trademark, and+          attribution notices from the Source form of the Work,+          excluding those notices that do not pertain to any part of+          the Derivative Works; and++      (d) If the Work includes a "NOTICE" text file as part of its+          distribution, then any Derivative Works that You distribute must+          include a readable copy of the attribution notices contained+          within such NOTICE file, excluding those notices that do not+          pertain to any part of the Derivative Works, in at least one+          of the following places: within a NOTICE text file distributed+          as part of the Derivative Works; within the Source form or+          documentation, if provided along with the Derivative Works; or,+          within a display generated by the Derivative Works, if and+          wherever such third-party notices normally appear. The contents+          of the NOTICE file are for informational purposes only and+          do not modify the License. You may add Your own attribution+          notices within Derivative Works that You distribute, alongside+          or as an addendum to the NOTICE text from the Work, provided+          that such additional attribution notices cannot be construed+          as modifying the License.++      You may add Your own copyright statement to Your modifications and+      may provide additional or different license terms and conditions+      for use, reproduction, or distribution of Your modifications, or+      for any such Derivative Works as a whole, provided Your use,+      reproduction, and distribution of the Work otherwise complies with+      the conditions stated in this License.++   5. Submission of Contributions. Unless You explicitly state otherwise,+      any Contribution intentionally submitted for inclusion in the Work+      by You to the Licensor shall be under the terms and conditions of+      this License, without any additional terms or conditions.+      Notwithstanding the above, nothing herein shall supersede or modify+      the terms of any separate license agreement you may have executed+      with Licensor regarding such Contributions.++   6. Trademarks. This License does not grant permission to use the trade+      names, trademarks, service marks, or product names of the Licensor,+      except as required for reasonable and customary use in describing the+      origin of the Work and reproducing the content of the NOTICE file.++   7. Disclaimer of Warranty. Unless required by applicable law or+      agreed to in writing, Licensor provides the Work (and each+      Contributor provides its Contributions) on an "AS IS" BASIS,+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+      implied, including, without limitation, any warranties or conditions+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+      PARTICULAR PURPOSE. You are solely responsible for determining the+      appropriateness of using or redistributing the Work and assume any+      risks associated with Your exercise of permissions under this License.++   8. Limitation of Liability. In no event and under no legal theory,+      whether in tort (including negligence), contract, or otherwise,+      unless required by applicable law (such as deliberate and grossly+      negligent acts) or agreed to in writing, shall any Contributor be+      liable to You for damages, including any direct, indirect, special,+      incidental, or consequential damages of any character arising as a+      result of this License or out of the use or inability to use the+      Work (including but not limited to damages for loss of goodwill,+      work stoppage, computer failure or malfunction, or any and all+      other commercial damages or losses), even if such Contributor+      has been advised of the possibility of such damages.++   9. Accepting Warranty or Additional Liability. While redistributing+      the Work or Derivative Works thereof, You may choose to offer,+      and charge a fee for, acceptance of support, warranty, indemnity,+      or other liability obligations and/or rights consistent with this+      License. However, in accepting such obligations, You may act only+      on Your own behalf and on Your sole responsibility, not on behalf+      of any other Contributor, and only if You agree to indemnify,+      defend, and hold each Contributor harmless for any liability+      incurred by, or claims asserted against, such Contributor by reason+      of your accepting any such warranty or additional liability.++   END OF TERMS AND CONDITIONS++   APPENDIX: How to apply the Apache License to your work.++      To apply the Apache License to your work, attach the following+      boilerplate notice, with the fields enclosed by brackets "[]"+      replaced with your own identifying information. (Don't include+      the brackets!)  The text should be enclosed in the appropriate+      comment syntax for the file format. We also recommend that a+      file or class name and description of purpose be included on the+      same "printed page" as the copyright notice for easier+      identification within third-party archives.++   Copyright [yyyy] [name of copyright owner]++   Licensed under the Apache License, Version 2.0 (the "License");+   you may not use this file except in compliance with the License.+   You may obtain a copy of the License at++       http://www.apache.org/licenses/LICENSE-2.0++   Unless required by applicable law or agreed to in writing, software+   distributed under the License is distributed on an "AS IS" BASIS,+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+   See the License for the specific language governing permissions and+   limitations under the License.

@@ -0,0 +1,2 @@+import Distribution.Simple+main = defaultMain

@@ -0,0 +1,45 @@+name:                hadoop-tools+version:             0.2+synopsis:            Tools for working with Hadoop.+description:         Tools for working with Hadoop.+license:             Apache-2.0+license-file:        LICENSE+author:              Jacob Stanley, Conrad Parker+maintainer:          Jacob Stanley <jacob@stanley.io>+category:            Data+build-type:          Simple+cabal-version:       >= 1.10++executable hh+  default-language: Haskell2010++  main-is: Main.hs+  hs-source-dirs: src++  other-modules:+    Paths_hadoop_tools++  build-depends:+      base                 >= 4.7 && < 5+    , boxes                >= 0.1+    , bytestring           >= 0.10+    , cereal               >= 0.4+    , cereal-conduit       >= 0.7+    , conduit              >= 1.2+    , conduit-extra        >= 1.1+    , configurator         >= 0.3+    , directory            >= 1.2+    , filepath             >= 1.3+    , network              >= 2.5+    , network-simple       >= 0.4+    , old-locale           >= 1.0+    , optparse-applicative >= 0.10+    , protobuf             >= 0.2.0.4+    , socks                >= 0.5+    , split                >= 0.2+    , text                 >= 1.1+    , time                 >= 1.4+    , transformers         >= 0.4+    , unix                 >= 2.7+    , unordered-containers >= 0.2+    , xmlhtml              >= 0.2.3.2

@@ -0,0 +1,436 @@+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RecordWildCards #-}++module Main (main) where++import           Control.Exception (SomeException, handle, throwIO)+import           Control.Monad+import           Control.Monad.IO.Class (MonadIO, liftIO)++import           Data.Bits ((.&.), shiftR)+import qualified Data.ByteString.Char8 as B+import           Data.Foldable (foldMap)+import qualified Data.HashMap.Lazy as H+import           Data.List (intercalate)+import           Data.List (isPrefixOf)+import           Data.List.Split (splitOn)+import           Data.Maybe (fromMaybe, maybeToList, mapMaybe, listToMaybe)+import           Data.Text (Text)+import qualified Data.Text as T+import qualified Data.Text.Encoding as T+import qualified Data.Text.IO as T+import qualified Data.Text.Read as T+import           Data.Time+import           Data.Time.Clock.POSIX+import           Data.Word (Word16, Word32, Word64)++import qualified Data.Configurator as C+import           Data.Configurator.Types (Worth(..))+import           System.Environment (getEnv, lookupEnv)+import           System.FilePath.Posix+import           System.IO.Unsafe (unsafePerformIO)+import           System.Locale (defaultTimeLocale)+import           System.Posix.User (getEffectiveUserName)+import           Text.PrettyPrint.Boxes hiding ((<>))+import           Text.XmlHtml++import           Data.Conduit (handleC)+import           Data.ProtocolBuffers+import           Data.ProtocolBuffers.Orphans ()++import           Hadoop.Protobuf.Hdfs+import           Hadoop.Rpc++import           Options.Applicative hiding (Success)++import           Data.Version (showVersion)+import           Paths_hadoop_tools (version)++------------------------------------------------------------------------++main :: IO ()+main = do+    cmd <- execParser optsParser+    handle printError (runRemote cmd)+  where+    optsParser = info (helper <*> options)+                      (fullDesc <> header "hh - Blazing fast interaction with HDFS")++runRemote :: Remote a -> IO a+runRemote remote = do+    hdfsUser   <- getHdfsUser+    nameNode   <- getNameNode+    socksProxy <- getSocksProxy++    let run = maybe runTcp runSocks socksProxy+    run nameNode hdfsUser remote++------------------------------------------------------------------------++configPath :: FilePath+configPath = unsafePerformIO $ do+    home <- getEnv "HOME"+    return (home </> ".hh")+{-# NOINLINE configPath #-}++getHdfsUser :: IO User+getHdfsUser = attempt [fromEnv, fromCfg] fromUnix+  where+    fromEnv :: IO (Maybe User)+    fromEnv  = fmap T.pack <$> lookupEnv "HADOOP_USER_NAME"++    fromCfg :: IO (Maybe User)+    fromCfg  = C.load [Optional configPath] >>= flip C.lookup "hdfs.user"++    fromUnix :: IO User+    fromUnix = T.pack <$> getEffectiveUserName++getNameNode :: IO NameNode+getNameNode = attempt [fromHadoopCfg, fromUserCfg] explode+  where+    fromHadoopCfg :: IO (Maybe NameNode)+    fromHadoopCfg = listToMaybe . maybe [] id <$> readHadoopNameNodes++    fromUserCfg :: IO (Maybe NameNode)+    fromUserCfg = do+        cfg  <- C.load [Optional configPath]+        host <- C.lookup cfg "namenode.host"+        port <- C.lookupDefault 8020 cfg "namenode.port"+        return (Endpoint <$> host <*> pure port)++    explode :: IO a+    explode = error $ "could not find namenode details in /etc/hadoop/conf/ or " <> configPath++getSocksProxy :: IO (Maybe SocksProxy)+getSocksProxy = do+    cfg   <- C.load [Optional configPath]+    mhost <- C.lookup cfg "proxy.host"+    case mhost of+        Nothing   -> return Nothing+        Just host -> Just . Endpoint host <$> C.lookupDefault 1080 cfg "proxy.port"++attempt :: Monad m => [m (Maybe a)] -> m a -> m a+attempt [] def     = def+attempt (io:ios) def = io >>= \mx -> case mx of+    Just x  -> return x+    Nothing -> attempt ios def++------------------------------------------------------------------------++type HadoopConfig = H.HashMap Text Text++readHadoopNameNodes :: IO (Maybe [NameNode])+readHadoopNameNodes = do+    cfg <- H.union <$> readHadoopConfig "/etc/hadoop/conf/core-site.xml"+                   <*> readHadoopConfig "/etc/hadoop/conf/hdfs-site.xml"+    return $ resolveNameNode cfg <$> (stripProto =<< H.lookup fsDefaultNameKey cfg)+  where+    proto            = "hdfs://"+    fsDefaultNameKey = "fs.defaultFS"+    nameNodesPrefix  = "dfs.ha.namenodes."+    rpcAddressPrefix = "dfs.namenode.rpc-address."++    stripProto :: Text -> Maybe Text+    stripProto uri | proto `T.isPrefixOf` uri = Just (T.drop (T.length proto) uri)+                   | otherwise                = Nothing++    resolveNameNode :: HadoopConfig -> Text -> [NameNode]+    resolveNameNode cfg name = case parseEndpoint name of+        Just ep -> [ep] -- contains "host:port" directly+        Nothing -> mapMaybe (\nn -> lookupAddress cfg $ name <> "." <> nn)+                            (lookupNameNodes cfg name)++    lookupNameNodes :: HadoopConfig -> Text -> [Text]+    lookupNameNodes cfg name = maybe [] id+                             $ T.splitOn "," <$> H.lookup (nameNodesPrefix <> name) cfg++    lookupAddress :: HadoopConfig -> Text -> Maybe Endpoint+    lookupAddress cfg name = parseEndpoint =<< H.lookup (rpcAddressPrefix <> name) cfg++    parseEndpoint :: Text -> Maybe Endpoint+    parseEndpoint ep = Endpoint host <$> port+      where+        host = T.takeWhile (/= ':') ep+        port = either (const Nothing) (Just . fst)+             $ T.decimal $ T.drop (T.length host + 1) ep++readHadoopConfig :: FilePath -> IO HadoopConfig+readHadoopConfig path = do+    exml <- parseXML path <$> B.readFile path+    case exml of+      Left  _   -> return H.empty+      Right xml -> return (toHashMap (docContent xml))+  where+    toHashMap = H.fromList . mapMaybe fromNode+              . concatMap (descendantElementsTag "property")++    fromNode n = (,) <$> (nodeText <$> childElementTag "name" n)+                     <*> (nodeText <$> childElementTag "value" n)++------------------------------------------------------------------------++workingDirConfigPath :: FilePath+workingDirConfigPath = unsafePerformIO $ do+    home <- getEnv "HOME"+    return (home </> ".hhwd")+{-# NOINLINE workingDirConfigPath #-}++getDefaultWorkingDir :: MonadIO m => m FilePath+getDefaultWorkingDir = liftIO $ (("/user" </>) . T.unpack) <$> getHdfsUser++getWorkingDir :: MonadIO m => m FilePath+getWorkingDir = liftIO $ handle onError+                       $ T.unpack . T.takeWhile (/= '\n')+                     <$> T.readFile workingDirConfigPath+  where+    onError :: SomeException -> IO FilePath+    onError = const getDefaultWorkingDir++setWorkingDir :: MonadIO m => FilePath -> m ()+setWorkingDir path = liftIO $ T.writeFile workingDirConfigPath+                            $ T.pack path <> "\n"++getAbsolute :: MonadIO m => FilePath -> m FilePath+getAbsolute path = liftIO (normalizePath <$> getPath)+  where+    getPath = if "/" `isPrefixOf` path+              then return path+              else getWorkingDir >>= \pwd -> return (pwd </> path)++normalizePath :: FilePath -> FilePath+normalizePath = intercalate "/" . dropAbsParentDir . splitOn "/"++dropAbsParentDir :: [FilePath] -> [FilePath]+dropAbsParentDir []       = error "dropAbsParentDir: not an absolute path"+dropAbsParentDir (p : ps) = p : (reverse $ fst $ go [] ps)+  where+    go []       (".." : ys) = go [] ys+    go (_ : xs) (".." : ys) = go xs ys+    go xs       (y    : ys) = go (y : xs) ys+    go xs       []          = (xs, [])++------------------------------------------------------------------------++data SubCommand = SubCommand+    { subName :: String+    , subDescription :: String+    , subMethod :: Parser (Remote ())+    }++sub :: SubCommand -> Mod CommandFields (Remote ())+sub SubCommand{..} = command subName (info subMethod $ progDesc subDescription)++options :: Parser (Remote ())+options = subparser (foldMap sub allSubCommands)++allSubCommands :: [SubCommand]+allSubCommands =+    [ subChDir+    , subDiskUsage+    , subList+    , subMkDir+    , subPwd+    , subRemove+    , subRename+    , subVersion+    ]++completePath :: Mod ArgumentFields a+completePath = completer (fileCompletion (const True)) <> metavar "PATH"++completeDir :: Mod ArgumentFields a+completeDir  = completer (fileCompletion (== Dir)) <> metavar "DIRECTORY"++subChDir :: SubCommand+subChDir = SubCommand "cd" "Change working directory" go+  where+    go = cd <$> optional (argument str (completeDir <> help "the directory to change to"))+    cd mpath = do+        path <- getAbsolute =<< maybe getDefaultWorkingDir return mpath+        mls <- getListing path+        liftIO $ case mls of+            Nothing -> putStrLn $ "Directory does not exist: " <> path+            Just _  -> setWorkingDir path++subDiskUsage :: SubCommand+subDiskUsage = SubCommand "du" "Show the amount of space used by file or directory" go+  where+    go = du <$> optional (argument str (completePath <> help "the file/directory to check the usage of"))+    du path = printDiskUsage =<< getAbsolute (maybe "" id path)++subList :: SubCommand+subList = SubCommand "ls" "List the contents of a directory" go+  where+    go = ls <$> (optional (argument str (completeDir <> help "the directory to list")))+    ls path = printListing =<< getAbsolute (fromMaybe "" path)++subMkDir :: SubCommand+subMkDir = SubCommand "mkdir" "Create a directory in the specified location" go+  where+    go = mkdir <$> argument str (completeDir       <> help "the directory to create")+               <*> switch       (short 'p' <> help "create intermediate directories")+    mkdir path parent =  do+      absPath <- getAbsolute path+      ok <- mkdirs absPath parent+      unless ok $ liftIO . putStrLn $ "Failed to create: " <> absPath++subPwd :: SubCommand+subPwd = SubCommand "pwd" "Print working directory" go+  where+    go = pure pwd+    pwd = liftIO . putStrLn =<< getWorkingDir++subRemove :: SubCommand+subRemove = SubCommand "rm" "Delete a file or directory" go+  where+    go = rm <$> argument str (completePath      <> help "the file/directory to remove")+            <*> switch       (short 'r' <> help "recursively remove the whole file hierarchy")+    rm path recursive = do+      absPath <- getAbsolute path+      ok <- delete absPath recursive+      unless ok $ liftIO . putStrLn $ "Failed to remove: " <> absPath++subRename :: SubCommand+subRename = SubCommand "mv" "Rename a file or directory" go+  where+    go = mv <$> argument str (completePath      <> help "source file/directory")+            <*> argument str (completePath      <> help "destination file/directory")+            <*> switch       (short 'f' <> help "overwrite destination if it exists")+    mv src dst force = do+      absSrc <- getAbsolute src+      absDst <- getAbsolute dst+      rename absSrc absDst force++subVersion :: SubCommand+subVersion = SubCommand "version" "Show version information" go+  where+    go = pure $ liftIO $ putStrLn $ "hh version " <> showVersion version++------------------------------------------------------------------------++fileCompletion :: (FileType -> Bool) -> Completer+fileCompletion p = mkCompleter $ \path -> handle ignore $ runRemote $ do+    let (dir, _) = splitFileName' path+    ls <- getListing =<< getAbsolute dir++    return $ filter (path `isPrefixOf`)+           . map (displayPath dir)+           . filter (p . get fsFileType)+           . concatMap (get dlPartialListing)+           . maybeToList  $ ls+  where+    ignore (RemoteError _ _) = return []++    splitFileName' x = case splitFileName x of+        ("./", f) -> ("", f)+        (d, f)    -> (d, f)++displayPath :: FilePath -> FileStatus -> FilePath+displayPath parent file = parent </> B.unpack (get fsPath file) <> suffix+  where+    suffix = case get fsFileType file of+        Dir -> "/"+        _   -> ""++------------------------------------------------------------------------++printDiskUsage :: FilePath -> Remote ()+printDiskUsage path = do+    mls <- getListing path+    case mls of+      Nothing -> liftIO $ putStrLn $ "File/directory does not exist: " <> path+      Just ls -> do+        let files = map (displayPath path) (get dlPartialListing ls)+        css <- zip files <$> mapM getDirSize files++        let col a f = vcat a (map (text . f) css)++        liftIO $ do+            printBox $ col right snd+                   <+> col left  fst+  where+    getDirSize f = handleC (\e -> if isAccessDenied e then return "-" else liftIO (throwIO e))+                           (formatSize . get csLength <$> getContentSummary f)++printListing :: FilePath -> Remote ()+printListing path = do+    mls <- getListing path+    case mls of+      Nothing -> liftIO . putStrLn $ "Directory does not exist: " <> path+      Just ls -> do+        let getPerms     = fromIntegral . get fpPerm . get fsPermission+            getBlockRepl = fromMaybe 0 . get fsBlockReplication++            hdfs2utc ms  = posixSecondsToUTCTime (fromIntegral ms / 1000)+            getModTime   = hdfs2utc . get fsModificationTime++            -- TODO: Fetch rest of partial listing+            xs      = get dlPartialListing ls+            col a f = vcat a (map (text . f) xs)++        liftIO $ do+            putStrLn $ "Found " <> show (length xs) <> " items"++            printBox $ col left  (\x -> formatMode (get fsFileType x) (getPerms x))+                   <+> col right (formatBlockRepl . getBlockRepl)+                   <+> col left  (T.unpack . get fsOwner)+                   <+> col left  (T.unpack . get fsGroup)+                   <+> col right (formatSize . get fsLength)+                   <+> col right (formatUTC . getModTime)+                   <+> col left  (T.unpack . T.decodeUtf8 . get fsPath)++------------------------------------------------------------------------++get :: HasField a => (t -> a) -> t -> FieldType a+get f x = getField (f x)++------------------------------------------------------------------------++type Perms = Word16++formatSize :: Word64 -> String+formatSize b | b <= 0            = "0"+             | b < 1000          = show b <> "B"+             | b < 1000000       = show (b `div` 1000) <> "K"+             | b < 1000000000    = show (b `div` 1000000) <> "M"+             | b < 1000000000000 = show (b `div` 1000000000) <> "G"+             | otherwise         = show (b `div` 1000000000000) <> "T"++formatBlockRepl :: Word32 -> String+formatBlockRepl x | x == 0    = "-"+                  | otherwise = show x++formatUTC :: UTCTime -> String+formatUTC = formatTime defaultTimeLocale "%Y-%m-%d %H:%M"++formatMode :: FileType -> Perms -> String+formatMode File    = ("-" <>) . formatPerms+formatMode Dir     = ("d" <>) . formatPerms+formatMode SymLink = ("l" <>) . formatPerms++formatPerms :: Perms -> String+formatPerms perms = format (perms `shiftR` 6)+                 <> format (perms `shiftR` 3)+                 <> format perms+  where+    format p = conv 0x4 "r" p+            <> conv 0x2 "w" p+            <> conv 0x1 "x" p++    conv bit rep p | (p .&. bit) /= 0 = rep+                   | otherwise        = "-"++------------------------------------------------------------------------++printError :: RemoteError -> IO ()+printError (RemoteError subject body)+    | oneLiner  = T.putStrLn firstLine+    | otherwise = T.putStrLn subject >> T.putStrLn body+  where+    oneLiner  = subject `elem` [ "org.apache.hadoop.security.AccessControlException"+                               , "org.apache.hadoop.fs.FileAlreadyExistsException"+                               , "java.io.FileNotFoundException" ]+    firstLine = T.takeWhile (/= '\n') body++isAccessDenied :: RemoteError -> Bool+isAccessDenied (RemoteError s _) = s == "org.apache.hadoop.security.AccessControlException"