Class BlockUploader

java.lang.Object
uk.ac.starlink.ttools.cone.BlockUploader

public class BlockUploader extends Object
Works with an UploadMatcher dividing the input table into chunks and uploading them separately to produce an arbitrarily large result while each upload/match operation is of a limited size.
Since:
15 May 2014
Author:
Mark Taylor
  • Constructor Summary

    Constructors
    Constructor
    Description
    BlockUploader(UploadMatcher umatcher, int blocksize, long maxrec, String outName, uk.ac.starlink.table.JoinFixAction uploadFixAct, uk.ac.starlink.table.JoinFixAction remoteFixAct, ServiceFindMode serviceMode, boolean oneToOne, boolean uploadEmpty)
    Constructor.
  • Method Summary

    Modifier and Type
    Method
    Description
    uk.ac.starlink.table.StarTable
    runMatch(uk.ac.starlink.table.StarTable inTable, QuerySequenceFactory qsFact, uk.ac.starlink.table.StoragePolicy storage)
    Performs an upload join in blocks.
    void
    setTruncationAdvice(String truncationAdvice)
    Sets a string that can be issued to the user as additional advice if there are truncations in block results.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Constructor Details

    • BlockUploader

      public BlockUploader(UploadMatcher umatcher, int blocksize, long maxrec, String outName, uk.ac.starlink.table.JoinFixAction uploadFixAct, uk.ac.starlink.table.JoinFixAction remoteFixAct, ServiceFindMode serviceMode, boolean oneToOne, boolean uploadEmpty)
      Constructor.
      Parameters:
      umatcher - upload matcher
      blocksize - maximum number of rows per uploaded block
      maxrec - maximum number of output rows accepted in total
      outName - name of output table
      uploadFixAct - name deduplication policy for upload table
      remoteFixAct - name deduplication policy for remote table
      serviceMode - upload match mode
      oneToOne - true iff output rows match 1:1 with input rows
      uploadEmpty - determines behaviour if there are no input rows: true means attempt the match anyway, false means throw an IOException
  • Method Details

    • runMatch

      public uk.ac.starlink.table.StarTable runMatch(uk.ac.starlink.table.StarTable inTable, QuerySequenceFactory qsFact, uk.ac.starlink.table.StoragePolicy storage) throws IOException
      Performs an upload join in blocks.

      As currently implemented, the input table needs to have random access, and the output table has random access. It would be possible to stream on input and output to some extent, though each input chunk will still have to be random access.

      The row indices given by ConeQueryRowSequence.getIndex calls associated with the supplied qsFact object must correspond to the row indices in the supplied inTable.

      Parameters:
      inTable - input table, must have random access
      qsFact - object to generate positional queries when applied to a table
      storage - storage policy for storing raw result table
      Throws:
      IOException
    • setTruncationAdvice

      public void setTruncationAdvice(String truncationAdvice)
      Sets a string that can be issued to the user as additional advice if there are truncations in block results.
      Parameters:
      truncationAdvice - user-readable advice string