|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.filecache.TrackerDistributedCacheManager
public class TrackerDistributedCacheManager
Manages a single machine's instance of a cross-job cache. This class would typically be instantiated by a TaskTracker (or something that emulates it, like LocalJobRunner). This class is internal to Hadoop, and should not be treated as a public interface.
Constructor Summary | |
---|---|
TrackerDistributedCacheManager(Configuration conf,
TaskController controller)
|
Method Summary | |
---|---|
static void |
createAllSymlink(Configuration conf,
File jobCacheDir,
File workDir)
This method create symlinks for all files in a given dir in another directory. |
static void |
determineCacheVisibilities(Configuration job)
Determines the visibilities of the distributed cache files and archives. |
static void |
determineTimestamps(Configuration job)
Determines timestamps of files to be cached, and stores those in the configuration. |
static long |
downloadCacheObject(Configuration conf,
URI source,
Path destination,
long desiredTimestamp,
boolean isArchive,
FsPermission permission)
Download a given path to the local file system. |
static boolean[] |
getArchiveVisibilities(Configuration conf)
Get the booleans on whether the archives are public or not. |
static void |
getDelegationTokens(Configuration job,
Credentials credentials)
For each archive or cache file - get the corresponding delegation token |
static boolean[] |
getFileVisibilities(Configuration conf)
Get the booleans on whether the files are public or not. |
TaskDistributedCacheManager |
newTaskDistributedCacheManager(JobID jobId,
Configuration taskConf)
|
void |
purgeCache()
Clear the entire contents of the cache and delete the backing files. |
void |
setArchiveSizes(JobID jobId,
long[] sizes)
|
static void |
validate(Configuration conf)
This is part of the framework API. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public TrackerDistributedCacheManager(Configuration conf, TaskController controller) throws IOException
IOException
Method Detail |
---|
public static long downloadCacheObject(Configuration conf, URI source, Path destination, long desiredTimestamp, boolean isArchive, FsPermission permission) throws IOException
conf
- the job's configurationsource
- the source to copy fromdestination
- where to copy the file. must be local fsdesiredTimestamp
- the required modification timestamp of the sourceisArchive
- is this an archive that should be expandedpermission
- the desired permissions of the file.
IOException
public static void createAllSymlink(Configuration conf, File jobCacheDir, File workDir) throws IOException
conf
- the configurationjobCacheDir
- the target directory for creating symlinksworkDir
- the directory in which the symlinks are created
IOException
public void purgeCache()
public TaskDistributedCacheManager newTaskDistributedCacheManager(JobID jobId, Configuration taskConf) throws IOException
IOException
public void setArchiveSizes(JobID jobId, long[] sizes) throws IOException
IOException
public static void determineTimestamps(Configuration job) throws IOException
job
- Configuration of a job.
IOException
public static void determineCacheVisibilities(Configuration job) throws IOException
job
-
IOException
public static boolean[] getFileVisibilities(Configuration conf)
conf
- The configuration which stored the timestamps
IOException
public static boolean[] getArchiveVisibilities(Configuration conf)
conf
- The configuration which stored the timestamps
public static void getDelegationTokens(Configuration job, Credentials credentials) throws IOException
job
- credentials
-
IOException
public static void validate(Configuration conf) throws InvalidJobConfException
conf
- a Configuration
to be cheked for duplication
in cached URIs
InvalidJobConfException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |