|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.mapreduce.OutputCommitter
org.apache.hadoop.mapred.OutputCommitter
org.apache.hadoop.mapred.FileOutputCommitter
@InterfaceAudience.Public @InterfaceStability.Stable public class FileOutputCommitter
An OutputCommitter that commits files specified
in job output directory i.e. ${mapreduce.output.fileoutputformat.outputdir}.
| Field Summary | |
|---|---|
static org.apache.commons.logging.Log |
LOG
|
static String |
SUCCEEDED_FILE_NAME
|
static String |
TEMP_DIR_NAME
Temporary directory name |
| Constructor Summary | |
|---|---|
FileOutputCommitter()
|
|
| Method Summary | |
|---|---|
void |
abortJob(JobContext context,
int runState)
For aborting an unsuccessful job's output. |
void |
abortTask(TaskAttemptContext context)
Discard the task output |
void |
cleanupJob(JobContext context)
Deprecated. |
void |
commitJob(JobContext context)
For committing job's output after successful job completion. |
void |
commitTask(TaskAttemptContext context)
To promote the task's temporary output to final output location The task's output is moved to the job's output directory. |
Path |
getWorkPath(TaskAttemptContext context,
Path outputPath)
|
boolean |
isRecoverySupported()
This method implements the new interface by calling the old method. |
boolean |
needsTaskCommit(TaskAttemptContext context)
Check whether task needs a commit |
void |
recoverTask(TaskAttemptContext context)
Recover the task output. |
void |
setupJob(JobContext context)
For the framework to setup the job output during initialization |
void |
setupTask(TaskAttemptContext context)
Sets up output for the task. |
| Methods inherited from class org.apache.hadoop.mapred.OutputCommitter |
|---|
abortJob, abortTask, cleanupJob, commitJob, commitTask, needsTaskCommit, recoverTask, setupJob, setupTask |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
public static final org.apache.commons.logging.Log LOG
public static final String TEMP_DIR_NAME
public static final String SUCCEEDED_FILE_NAME
| Constructor Detail |
|---|
public FileOutputCommitter()
| Method Detail |
|---|
public Path getWorkPath(TaskAttemptContext context,
Path outputPath)
throws IOException
IOException
public void setupJob(JobContext context)
throws IOException
OutputCommitter
setupJob in class OutputCommittercontext - Context of the job whose output is being written.
IOException - if temporary output could not be created
public void commitJob(JobContext context)
throws IOException
OutputCommitter
commitJob in class OutputCommittercontext - Context of the job whose output is being written.
IOException
@Deprecated
public void cleanupJob(JobContext context)
throws IOException
OutputCommitter
cleanupJob in class OutputCommittercontext - Context of the job whose output is being written.
IOException
public void abortJob(JobContext context,
int runState)
throws IOException
OutputCommitterJobStatus.FAILED or
JobStatus.KILLED
abortJob in class OutputCommittercontext - Context of the job whose output is being written.runState - final runstate of the job
IOException
public void setupTask(TaskAttemptContext context)
throws IOException
OutputCommitter
setupTask in class OutputCommittercontext - Context of the task whose output is being written.
IOException
public void commitTask(TaskAttemptContext context)
throws IOException
OutputCommitter
commitTask in class OutputCommittercontext - Context of the task whose output is being written.
IOException - if commit is not
public void abortTask(TaskAttemptContext context)
throws IOException
OutputCommitter
abortTask in class OutputCommitterIOException
public boolean needsTaskCommit(TaskAttemptContext context)
throws IOException
OutputCommitter
needsTaskCommit in class OutputCommitterIOExceptionpublic boolean isRecoverySupported()
OutputCommitter
isRecoverySupported in class OutputCommittertrue if task output recovery is supported,
false otherwiseOutputCommitter.recoverTask(TaskAttemptContext)
public void recoverTask(TaskAttemptContext context)
throws IOException
OutputCommitterMRConstants.APPLICATION_ATTEMPT_ID key in
JobContext.getConfiguration() for the
OutputCommitter.
If an exception is thrown the task will be attempted again.
recoverTask in class OutputCommittercontext - Context of the task whose output is being recovered
IOException
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||