|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.conf.Configured
org.apache.hadoop.fs.FileSystem
org.apache.hadoop.fs.FilterFileSystem
@InterfaceAudience.Public @InterfaceStability.Stable public class FilterFileSystem
A FilterFileSystem contains
some other file system, which it uses as
its basic file system, possibly transforming
the data along the way or providing additional
functionality. The class FilterFileSystem
itself simply overrides all methods of
FileSystem with versions that
pass all requests to the contained file
system. Subclasses of FilterFileSystem
may further override some of these methods
and may also provide additional methods
and fields.
| Field Summary | |
|---|---|
protected FileSystem |
fs
|
| Fields inherited from class org.apache.hadoop.fs.FileSystem |
|---|
DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, statistics |
| Constructor Summary | |
|---|---|
FilterFileSystem()
|
|
FilterFileSystem(FileSystem fs)
|
|
| Method Summary | |
|---|---|
FSDataOutputStream |
append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation). |
protected void |
checkPath(Path path)
Check that a Path belongs to this FileSystem. |
void |
close()
No more filesystem operations are needed. |
void |
completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Called when we're all done writing to the target. |
void |
copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk. |
void |
copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
The src file is on the local disk. |
void |
copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is on the local disk. |
void |
copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk. |
FSDataOutputStream |
create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress reporting. |
boolean |
delete(Path f,
boolean recursive)
Delete a file |
boolean |
deleteOnExit(Path f)
Mark a path to be deleted when FileSystem is closed. |
String |
getCanonicalServiceName()
Get a canonical service name for this file system. |
protected URI |
getCanonicalUri()
Returns a qualified URI whose scheme and authority identify this FileSystem. |
Configuration |
getConf()
Return the configuration used by this object. |
long |
getDefaultBlockSize()
Return the number of bytes that large input files should be optimally be split into to minimize i/o time. |
short |
getDefaultReplication()
Get the default replication. |
org.apache.hadoop.security.token.Token<?> |
getDelegationToken(String renewer)
Deprecated - use @link FileSystem.getDelegationTokens(String)
Get a new delegation token for this file system. |
List<org.apache.hadoop.security.token.Token<?>> |
getDelegationTokens(String renewer)
Get one or more delegation tokens associated with the filesystem. |
List<org.apache.hadoop.security.token.Token<?>> |
getDelegationTokens(String renewer,
org.apache.hadoop.security.Credentials credentials)
|
BlockLocation[] |
getFileBlockLocations(FileStatus file,
long start,
long len)
Return an array containing hostnames, offset and size of portions of the given file. |
FileChecksum |
getFileChecksum(Path f)
Get the checksum of a file. |
FileStatus |
getFileStatus(Path f)
Get file status. |
Path |
getHomeDirectory()
Return the current user's home directory in this filesystem. |
protected Path |
getInitialWorkingDirectory()
Note: with the new FilesContext class, getWorkingDirectory() will be removed. |
FileSystem |
getRawFileSystem()
Get the raw file system |
FsStatus |
getStatus(Path p)
Returns a status object describing the use and capacity of the file system. |
URI |
getUri()
Returns a URI whose scheme and authority identify this FileSystem. |
long |
getUsed()
Return the total size of all files in the filesystem. |
Path |
getWorkingDirectory()
Get the current working directory for the given file system |
void |
initialize(URI name,
Configuration conf)
Called after a new FileSystem instance is constructed. |
org.apache.hadoop.fs.RemoteIterator<Path> |
listCorruptFileBlocks(Path path)
|
org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus> |
listLocatedStatus(Path f)
List files and its block locations in a directory. |
FileStatus[] |
listStatus(Path f)
List files in a directory. |
Path |
makeQualified(Path path)
Make sure that a path specifies a FileSystem. |
boolean |
mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into directories. |
FSDataInputStream |
open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path. |
protected FSDataOutputStream |
primitiveCreate(Path f,
FsPermission absolutePermission,
EnumSet<CreateFlag> flag,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
int bytesPerChecksum)
|
protected boolean |
primitiveMkdir(Path f,
FsPermission abdolutePermission)
This version of the mkdirs method assumes that the permission is absolute. |
boolean |
rename(Path src,
Path dst)
Renames Path src to Path dst. |
Path |
resolvePath(Path p)
Return the fully-qualified path of path f resolving the path through any symlinks or mount point |
void |
setOwner(Path p,
String username,
String groupname)
Set owner of a path (i.e. |
void |
setPermission(Path p,
FsPermission permission)
Set permission of a path. |
boolean |
setReplication(Path src,
short replication)
Set replication for an existing file. |
void |
setTimes(Path p,
long mtime,
long atime)
Set access time of a file |
void |
setVerifyChecksum(boolean verifyChecksum)
Set the verify checksum flag. |
void |
setWorkingDirectory(Path newDir)
Set the current working directory for the given file system. |
Path |
startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to. |
| Methods inherited from class org.apache.hadoop.fs.FileSystem |
|---|
append, append, clearStatistics, closeAll, closeAllForUGI, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, delete, exists, get, get, get, getAllStatistics, getBlockSize, getContentSummary, getDefaultPort, getDefaultUri, getFileBlockLocations, getLength, getLocal, getName, getNamed, getReplication, getServerDefaults, getStatistics, getStatistics, getStatus, globStatus, globStatus, isDirectory, isFile, listFiles, listLocatedStatus, listStatus, listStatus, listStatus, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveMkdir, printStatistics, processDeleteOnExit, rename, setDefaultUri, setDefaultUri |
| Methods inherited from class org.apache.hadoop.conf.Configured |
|---|
setConf |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
protected FileSystem fs
| Constructor Detail |
|---|
public FilterFileSystem()
public FilterFileSystem(FileSystem fs)
| Method Detail |
|---|
public FileSystem getRawFileSystem()
public void initialize(URI name,
Configuration conf)
throws IOException
initialize in class FileSystemname - a uri whose authority section names the host, port, etc.
for this FileSystemconf - the configuration
IOExceptionpublic URI getUri()
getUri in class FileSystemprotected URI getCanonicalUri()
getCanonicalUri in class FileSystemNetUtils.getCanonicalUri(URI, int)public Path makeQualified(Path path)
makeQualified in class FileSystempath - to useprotected void checkPath(Path path)
checkPath in class FileSystempath - to check
public BlockLocation[] getFileBlockLocations(FileStatus file,
long start,
long len)
throws IOException
FileSystem
getFileBlockLocations in class FileSystemfile - FilesStatus to get data fromstart - offset into the given filelen - length for which to get locations for
IOException
public Path resolvePath(Path p)
throws IOException
FileSystem
resolvePath in class FileSystemp - path to be resolved
FileNotFoundException
IOException
public FSDataInputStream open(Path f,
int bufferSize)
throws IOException
open in class FileSystemf - the file name to openbufferSize - the size of the buffer to be used.
IOException
public FSDataOutputStream append(Path f,
int bufferSize,
Progressable progress)
throws IOException
append in class FileSystemf - the existing file to be appended.bufferSize - the size of the buffer to be used.progress - for reporting progress if it is not null.
IOException
public FSDataOutputStream create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
throws IOException
create in class FileSystemf - the file name to openoverwrite - if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize - the size of the buffer to be used.replication - required block replication for the file.
IOExceptionFileSystem.setPermission(Path, FsPermission)
public boolean setReplication(Path src,
short replication)
throws IOException
setReplication in class FileSystemsrc - file namereplication - new replication
IOException
public boolean rename(Path src,
Path dst)
throws IOException
rename in class FileSystemsrc - path to be renameddst - new path after rename
IOException - on failure
public boolean delete(Path f,
boolean recursive)
throws IOException
delete in class FileSystemf - the path to delete.recursive - if path is a directory and set to
true, the directory is deleted else throws an exception. In
case of a file the recursive can be set to either true or false.
IOException
public boolean deleteOnExit(Path f)
throws IOException
deleteOnExit in class FileSystemf - the path to delete.
IOException
public FileStatus[] listStatus(Path f)
throws IOException
listStatus in class FileSystemf - given path
FileNotFoundException - when the path does not exist;
IOException see specific implementation
IOException
public org.apache.hadoop.fs.RemoteIterator<Path> listCorruptFileBlocks(Path path)
throws IOException
listCorruptFileBlocks in class FileSystemIOException
public org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus> listLocatedStatus(Path f)
throws IOException
listLocatedStatus in class FileSystemf - is the path
FileNotFoundException - If f does not exist
IOException - If an I/O error occurredpublic Path getHomeDirectory()
FileSystem
getHomeDirectory in class FileSystempublic void setWorkingDirectory(Path newDir)
setWorkingDirectory in class FileSystemnewDir - public Path getWorkingDirectory()
getWorkingDirectory in class FileSystemprotected Path getInitialWorkingDirectory()
FileSystem
getInitialWorkingDirectory in class FileSystem
public FsStatus getStatus(Path p)
throws IOException
getStatus in class FileSystemp - Path for which status should be obtained. null means
the default partition.
IOException - see specific implementation
public boolean mkdirs(Path f,
FsPermission permission)
throws IOException
mkdirs in class FileSystemf - path to createpermission - to apply to f
IOException
public void copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
throws IOException
copyFromLocalFile in class FileSystemdelSrc - whether to delete the srcsrc - pathdst - path
IOException
public void copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
throws IOException
copyFromLocalFile in class FileSystemdelSrc - whether to delete the srcoverwrite - whether to overwrite an existing filesrcs - array of paths which are sourcedst - path
IOException
public void copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
throws IOException
copyFromLocalFile in class FileSystemdelSrc - whether to delete the srcoverwrite - whether to overwrite an existing filesrc - pathdst - path
IOException
public void copyToLocalFile(boolean delSrc,
Path src,
Path dst)
throws IOException
copyToLocalFile in class FileSystemdelSrc - whether to delete the srcsrc - pathdst - path
IOException
public Path startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
throws IOException
startLocalOutput in class FileSystemfsOutputFile - path of output filetmpLocalFile - path of local tmp file
IOException
public void completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
throws IOException
completeLocalOutput in class FileSystemfsOutputFile - path of output filetmpLocalFile - path to local tmp file
IOException
public long getUsed()
throws IOException
getUsed in class FileSystemIOExceptionpublic long getDefaultBlockSize()
getDefaultBlockSize in class FileSystempublic short getDefaultReplication()
getDefaultReplication in class FileSystem
public FileStatus getFileStatus(Path f)
throws IOException
getFileStatus in class FileSystemf - The path we want information from
FileNotFoundException - when the path does not exist;
IOException see specific implementation
IOException
public FileChecksum getFileChecksum(Path f)
throws IOException
getFileChecksum in class FileSystemf - The file path
IOExceptionpublic void setVerifyChecksum(boolean verifyChecksum)
setVerifyChecksum in class FileSystempublic Configuration getConf()
Configurable
getConf in interface ConfigurablegetConf in class Configured
public void close()
throws IOException
FileSystem
close in interface Closeableclose in class FileSystemIOException
public void setOwner(Path p,
String username,
String groupname)
throws IOException
setOwner in class FileSystemp - The pathusername - If it is null, the original username remains unchanged.groupname - If it is null, the original groupname remains unchanged.
IOException
public void setTimes(Path p,
long mtime,
long atime)
throws IOException
setTimes in class FileSystemp - The pathmtime - Set the modification time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set modification time.atime - Set the access time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set access time.
IOException
public void setPermission(Path p,
FsPermission permission)
throws IOException
setPermission in class FileSystemIOException
protected FSDataOutputStream primitiveCreate(Path f,
FsPermission absolutePermission,
EnumSet<CreateFlag> flag,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
int bytesPerChecksum)
throws IOException
primitiveCreate in class FileSystemIOException
protected boolean primitiveMkdir(Path f,
FsPermission abdolutePermission)
throws IOException
FileSystem
primitiveMkdir in class FileSystemIOExceptionpublic String getCanonicalServiceName()
FileSystem
getCanonicalServiceName in class FileSystemSecurityUtil.buildDTServiceName(URI, int)
public org.apache.hadoop.security.token.Token<?> getDelegationToken(String renewer)
throws IOException
FileSystemFileSystem.getDelegationTokens(String)
Get a new delegation token for this file system.
renewer - the account name that is allowed to renew the token.
IOException
public List<org.apache.hadoop.security.token.Token<?>> getDelegationTokens(String renewer)
throws IOException
FileSystem
renewer - the account name that is allowed to renew the token.
IOException
public List<org.apache.hadoop.security.token.Token<?>> getDelegationTokens(String renewer,
org.apache.hadoop.security.Credentials credentials)
throws IOException
renewer - the account name that is allowed to renew the token.credentials - a Credentials object containing already knowing
delegationTokens.
IOExceptionThis is similar to getDelegationTokens, with the added restriction that if
a token is already present in the passed Credentials object - that token
is returned instead of a new delegation token.
If the token is found to be cached in the Credentials object, this API does
not verify the token validity or the passed in renewer.
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||