Package 'rhdf5'

Description

R function to create an HDF5 attribute and defining its dimensionality.

Usage

h5createAttribute(
  obj,
  attr,
  dims,
  maxdims = dims,
  file,
  storage.mode = "double",
  H5type = NULL,
  size = NULL,
  encoding = NULL,
  cset = NULL,
  native = FALSE
)

Arguments

Details

Creates a new attribute and attaches it to an existing HDF5 object. The function will fail, if the file doesn't exist or if there exists already another attribute with the same name for this object.

You can use h5writeAttribute() immediately. It will create the attribute for you.

Value

Returns TRUE is attribute was created successfully and FALSE otherwise.

Author(s)

Bernd Fischer

References

https://portal.hdfgroup.org/display/HDF5

See Also

h5createFile(), h5createGroup(), h5createDataset(), h5read(), h5write(), rhdf5

Examples

h5File <- tempfile(pattern = "ex_createAttribute.h5")
h5createFile(h5File)
h5write(1:1, h5File, "A")
fid <- H5Fopen(h5File)
did <- H5Dopen(fid, "A")
h5createAttribute (did, "time", c(1,10))
H5Dclose(did)
H5Fclose(fid)

Description

R function to create an HDF5 dataset and defining its dimensionality and compression behaviour.

Usage

h5createDataset(
  file,
  dataset,
  dims,
  maxdims = dims,
  storage.mode = "double",
  H5type = NULL,
  size = NULL,
  encoding = NULL,
  chunk = dims,
  fillValue,
  level = 6,
  filter = "gzip",
  shuffle = TRUE,
  native = FALSE
)

Arguments

Details

Creates a new dataset in an existing HDF5 file. The function will fail if the file doesn't exist or if there exists already another dataset with the same name within the specified file.

The size argument is only used when storage.mode = 'character'. When storing strings HDF5 can use either a fixed or variable length datatype. Setting size to a positive integer will use fixed length strings where size defines the length. rhdf5 writes null padded strings by default and so to avoid data loss the value provided here should be the length of the longest string. Setting size = NULL will use variable length strings. The choice is probably dependent on the nature of the strings you're writing. The principle difference is that a dataset of variable length strings will not be compressed by HDF5 but each individual string only uses the space it requires, whereas in a fixed length dataset each string is of length uses size, but the whole dataset can be compressed. This explored more in the examples below.

The filter argument can take several options matching to compression filters distributed in either with the HDF5 library in Rhdf5lib or via the rhdf5filters package. The plugins available and the corresponding values for selecting them are shown below:

zlib: Ubiquitous deflate compression algorithm used in GZIP or ZIP files. All three options below achieve the same result.

• "GZIP",

• "ZLIB",

• "DEFLATE"

szip: Compression algorithm maintained by the HDF5 group.

• "SZIP"

bzip2

• "BZIP2"

BLOSC meta compressor: As a meta-compressor BLOSC wraps several different compression algorithms. Each of the options below will active a different compression filter.

• "BLOSC_BLOSCLZ"

• "BLOSC_LZ4"

• "BLOSC_LZ4HC"

• "BLOSC_SNAPPY"

• "BLOSC_ZLIB"

• "BLOSC_ZSTD"

lzf

• "LZF"

Disable: It is possible to write chunks without any compression applied.

• "NONE"

Value

Returns (invisibly) TRUE if dataset was created successfully and FALSE otherwise.

Author(s)

Bernd Fischer, Mike L. Smith

See Also

h5createFile(), h5createGroup(), h5read(), h5write()

Examples

h5File <- tempfile(pattern = "_ex_createDataset.h5")
h5createFile(h5File)

# create dataset with compression
h5createDataset(h5File, "A", c(5,8), storage.mode = "integer", chunk=c(5,1), level=6)

# create dataset without compression
h5createDataset(h5File, "B", c(5,8), storage.mode = "integer")
h5createDataset(h5File, "C", c(5,8), storage.mode = "double")

# create dataset with bzip2 compression
h5createDataset(h5File, "D", c(5,8), storage.mode = "integer",
    chunk=c(5,1), filter = "BZIP2", level=6)

# create a dataset of strings & define size based on longest string
ex_strings <- c('long', 'longer', 'longest')
h5createDataset(h5File, "E",
    storage.mode = "character", chunk = 3, level = 6,
    dims = length(ex_strings), size = max(nchar(ex_strings)))


# write data to dataset
h5write(matrix(1:40,nr=5,nc=8), file=h5File, name="A")
# write second column
h5write(matrix(1:5,nr=5,nc=1), file=h5File, name="B", index=list(NULL,2))
# write character vector
h5write(ex_strings, file = h5File, name = "E")

h5dump( h5File )

## Investigating fixed vs variable length string datasets

## create 1000 random strings with length between 50 and 100 characters
words <- ceiling(runif(n = 1000, min = 50, max = 100)) |>
vapply(FUN = \(x) {
 paste(sample(letters, size = x, replace = TRUE), collapse = "")
},
FUN.VALUE = character(1))

## create two HDF5 files
f1 <- tempfile()
f2 <- tempfile()
h5createFile(f1)
h5createFile(f2)

## create two string datasets
## the first is variable length strings, the second fixed at the length of our longest word
h5createDataset(f1, "strings", dims = length(words), storage.mode = "character", 
                size = NULL, chunk = 25)
h5createDataset(f2, "strings", dims = length(words), storage.mode = "character", 
                size = max(nchar(words)), chunk = 25)

## Write the data
h5write(words, f1, "strings")
h5write(words, f2, "strings")

## Check file sizes.
## In this example the fixed length string dataset is normally much smaller
file.size(f1)
file.size(f2)

Description

R function to create an empty HDF5 file.

Usage

h5createFile(file)

Arguments

Details

Creates an empty HDF5 file.

Value

Returns (invisibly) TRUE is file was created successfully and FALSE otherwise.

Author(s)

Bernd Fischer

See Also

h5createGroup(), h5createDataset(), h5read(), h5write(), rhdf5

Examples

h5File <- tempfile(pattern = "ex_createFile.h5")

h5createFile(h5File)

# create groups
h5createGroup(h5File,"foo")
h5createGroup(h5File,"foo/foobaa")

h5ls(h5File)

Description

Creates a group within an HDF5 file.

Usage

h5createGroup(file, group)

Arguments

Details

Creates a new group within an HDF5 file.

Value

Returns TRUE is group was created successfully and FALSE otherwise.

Author(s)

Bernd Fischer

See Also

h5createFile(), h5createDataset(), h5read(), h5write()

Examples

h5File <- tempfile(pattern = "ex_createGroup.h5")
h5createFile(h5File)

# create groups
h5createGroup(h5File, "foo")
h5createGroup(h5File, "foo/foobaa")

h5ls(h5File)

Description

Deletes the specified group or dataset from within an HDF5 file.

Usage

h5delete(file, name)

Arguments

Author(s)

Mike Smith

Description

Deletes an attribute associated with a group or dataset within an HDF5 file.

Usage

h5deleteAttribute(file, name, attribute)

Arguments

Author(s)

Mike Smith

Description

Dump the content of an HDF5 file.

Usage

h5dump(
  file,
  recursive = TRUE,
  load = TRUE,
  all = FALSE,
  index_type = h5default("H5_INDEX"),
  order = h5default("H5_ITER"),
  s3 = FALSE,
  s3credentials = NULL,
  ...,
  native = FALSE
)

Arguments

Value

Returns a hierarchical list structure representing the HDF5 group hierarchy. It either returns the datasets within the list structure (load=TRUE) or it returns a data.frame for each dataset with the dataset header information (load=FALSE).

Author(s)

Bernd Fischer, Mike L. Smith

See Also

h5ls()

Examples

h5File <- tempfile(pattern = "ex_dump.h5")
h5createFile(h5File)

# create groups
h5createGroup(h5File,"foo")
h5createGroup(h5File,"foo/foobaa")

# write a matrix
B = array(seq(0.1,2.0,by=0.1),dim=c(5,2,2))
attr(B, "scale") <- "liter"
h5write(B, h5File,"foo/B")

# list content of hdf5 file
h5dump(h5File)

# list content of an hdf5 file in a public S3 bucket

h5dump(file = "https://rhdf5-public.s3.eu-central-1.amazonaws.com/h5ex_t_array.h5", s3 = TRUE)

Description

Sets the options for handling HDF5 error messages in the R sessions.

Usage

h5errorHandling(type = "normal")

Arguments

Value

Returns 0 if options are set successfully.

Author(s)

Bernd Fischer

See Also

rhdf5

Examples

h5errorHandling("normal")

Description

HDF5 1.10 uses file locking by default. On some file systems this is not available, and the HDF5 library will throw an error if the user attempts to create or access a file located on such a file system. These functions help identify if file locking is available without throwing an error, and allow the locking to be disabled for the duration of the R session if needed.

Usage

h5testFileLocking(location)

h5disableFileLocking()

h5enableFileLocking()

Arguments

Details

h5testFileLocking will create a temporary file and then attempt to apply a file lock using the appropriate function within the HDF5 library. The success or failure of the locking is then recorded and the temporary file removed. Even relatively low level functions such as H5Fcreate will fail inelegantly if file locking fails.

h5disableFileLocking will set the environment variable RHDF5_USE_FILE_LOCKING=FALSE, which is the recommended was to disable this behaviour if file locking is not supported. This will only persist within the current R session. You can set the environment variable outside of R if this is a more general issue on your system.

h5enableFileLocking will unset the RHDF5_USE_FILE_LOCKING environment variable.

More discussion of HDF5's use of file locking can be found online e.g. https://forum.hdfgroup.org/t/hdf5-1-10-0-and-flock/3761/4 or https://forum.hdfgroup.org/t/hdf5-files-on-nfs/3985/5

Value

h5testFileLocking returns TRUE if a file can be successfully locked at the specified location, or FALSE otherwise.

h5disableFileLocking and h5enableFileLocking set are called for the side effect of setting or unsetting the environment variable HDF5_USE_FILE_LOCKING and do not return anything.

Author(s)

Mike Smith

Examples

## either a file name or directory can be tested
file <- tempfile()
dir <- tempdir()

h5testFileLocking(dir)
h5testFileLocking(file)

## we can check for file locking, and disable if needed
if( !h5testFileLocking(dir) ) {
  h5disableFileLocking()
}

Description

Reads objects in HDF5 files. This function can be used to read either full arrays/vectors or subarrays (hyperslabs) from an existing dataset.

Usage

h5read(
  file,
  name,
  index = NULL,
  start = NULL,
  stride = NULL,
  block = NULL,
  count = NULL,
  compoundAsDataFrame = TRUE,
  callGeneric = TRUE,
  read.attributes = FALSE,
  drop = FALSE,
  ...,
  native = FALSE,
  s3 = FALSE,
  s3credentials = NULL
)

Arguments

Details

Read an R object from an HDF5 file. If none of the arguments start, stride, block, count are specified, the dataset has the same dimension in the HDF5 file and in memory. If the dataset already exists in the HDF5 file, one can read subarrays, so called hyperslabs from the HDF5 file. The arguments start, stride, block, count define the subset of the dataset in the HDF5 file that is to be read/written. See these introductions to hyperslabs: https://support.hdfgroup.org/HDF5/Tutor/selectsimple.html, https://support.hdfgroup.org/HDF5/Tutor/select.html and http://ftp.hdfgroup.org/HDF5/Tutor/phypecont.html. Please note that in R the first dimension is the fastest changing dimension.

When viewing the HDF5 datasets with any C-program (e.g. HDFView), the order of dimensions is inverted. In the R interface counting starts with 1, whereas in the C-programs (e.g. HDFView) counting starts with 0.

Special cases. There are a few instances where rhdf5 will make assumptions about the dataset you are reading and treat it slightly differently. 1) complex numbers. If your datasets is a compound datatype, has only two columns, and these are named 'r' and 'i' rhdf5 will assume the data is intended to be complex numbers and will read this into R's complex type. If that is not the case, you will need to extract the two values separately using the Re() and Im() accessors manually.

Value

h5read returns an array with the data read.

Author(s)

Bernd Fischer, Mike Smith

See Also

h5ls

Examples

h5File <- tempfile(pattern = "ex_hdf5file.h5")
h5createFile(h5File)

# write a matrix
B = array(seq(0.1,2.0,by=0.1),dim=c(5,2,2))
h5write(B, h5File, "B")

# read a matrix
E = h5read(h5File,"B")

# write and read submatrix
h5createDataset(h5File, "S", c(5,8), storage.mode = "integer", chunk=c(5,1), level=7)
h5write(matrix(1:5,nr=5,nc=1), file=h5File, name="S", index=list(NULL,1))
h5read(h5File, "S")
h5read(h5File, "S", index=list(NULL,2:3))

# Read a subset of an hdf5 file in a public S3 bucket

h5read('https://rhdf5-public.s3.eu-central-1.amazonaws.com/rhdf5ex_t_float_3d.h5', 
      s3 = TRUE, name = "a1", index = list(NULL, 3, NULL))

Description

Read all attributes from a given location in an HDF5 file

Usage

h5readAttributes(file, name, native = FALSE, ...)

Arguments

Value

A named list of the same length as the number of attributes attached to the specific object. The names of the list entries correspond to the attribute names. If no attributes are found an empty list is returned.

Description

Saves a number of R objects to an HDF5 file.

Usage

h5save(..., file, name = NULL, createnewfile = TRUE, native = FALSE)

Arguments

Details

The objects will be saved to the HDF5 file. If the file does not exists it will be created. The data can be read again by either h5dump() or individually for each dataset by h5read().

Value

Nothing returned.

Author(s)

Bernd Fischer

See Also

h5ls(), h5write()

Examples

A = 1:7;  B = 1:18; D = seq(0,1,by=0.1)

h5File <- tempfile(pattern = "ex_save.h5")
h5save(A, B, D, file = h5File)
h5dump(h5File)

Description

Set a new dataset extension to an existing dataset in an HDF5 file

Usage

h5set_extent(file, dataset, dims, native = FALSE)

Arguments

Value

Returns TRUE if the dimension of the dataset was changed successfully and FALSE otherwise.

Author(s)

Bernd Fischer, Mike Smith

Examples

tmpfile <- tempfile()
h5createFile(file=tmpfile)
h5createDataset(tmpfile, "A", c(10,12), c(20,24))
h5ls(tmpfile, all=TRUE)[c("dim", "maxdim")]
h5set_extent(tmpfile, "A", c(20,24))
h5ls(tmpfile, all=TRUE)[c("dim", "maxdim")]

Description

Writes an R object to an HDF5 file. This function can be used to write either full arrays/vectors or subarrays (hyperslabs) within an existing dataset.

Usage

h5write(obj, file, name, ...)

## Default S3 method:
h5write(
  obj,
  file,
  name,
  createnewfile = TRUE,
  write.attributes = FALSE,
  ...,
  native = FALSE
)

h5writeDataset(obj, h5loc, name, ...)

## S3 method for class 'data.frame'
h5writeDataset(obj, h5loc, name, level = 6, chunk, DataFrameAsCompound = TRUE)

## S3 method for class 'array'
h5writeDataset(
  obj,
  h5loc,
  name,
  index = NULL,
  start = NULL,
  stride = NULL,
  block = NULL,
  count = NULL,
  size = NULL,
  variableLengthString = FALSE,
  encoding = NULL,
  level = 6
)

Arguments

Details

Writes an R object to an HDF5 file. If none of the arguments start, stride, block, count is specified, the dataset has the same dimension in the HDF5 file and in memory. If the dataset already exists in the HDF5 file, one can write subarrays, (so called hyperslabs) to the HDF5 file. The arguments start, stride, block, count define the subset of the dataset in the HDF5 file that is to be written to. See these introductions to hyperslabs: https://support.hdfgroup.org/HDF5/Tutor/selectsimple.html, https://support.hdfgroup.org/HDF5/Tutor/select.html and http://ftp.hdfgroup.org/HDF5/Tutor/phypecont.html. Please note that in R the first dimension is the fastest changing dimension.

When viewing the HDF5 datasets with any C-program (e.g. HDFView), the order of dimensions is inverted. In the R interface counting starts with 1, whereas in the C-programs (e.g. HDFView) counting starts with 0.

If code obj is of type 'complex' then it will be written as a compound datatype to the HDF5, with elements named 'r' and 'i' for the real and imaginary parts respectively.

Value

h5write returns 0 if successful.

Author(s)

Bernd Fischer, Mike Smith

References

https://portal.hdfgroup.org/display/HDF5

See Also

h5ls, h5createFile, h5createDataset, rhdf5

Examples

h5File <- tempfile(fileext = ".h5")
h5createFile( h5File )

# write a matrix
B = array(seq(0.1,2.0,by=0.1),dim=c(5,2,2))
attr(B, "scale") <- "liter"
h5write(B, h5File,"B")

# write a submatrix
h5createDataset(h5File, "S", c(5,8), storage.mode = "integer", chunk=c(5,1), level=7)
h5write(matrix(1:5,nr=5,nc=1), file=h5File, name="S", index=list(NULL,1))

Description

Write an R object as an HDF5 attribute

Usage

h5writeAttribute(
  attr,
  h5obj,
  name,
  h5loc,
  encoding = NULL,
  variableLengthString = FALSE,
  asScalar = FALSE,
  checkForNA = TRUE
)

## S3 method for class 'array'
h5writeAttribute(
  attr,
  h5obj,
  name,
  h5loc,
  encoding = NULL,
  variableLengthString = FALSE,
  asScalar = FALSE,
  checkForNA = TRUE
)

Arguments

Description

Close an HDF5 attribute

Usage

H5Aclose(h5attribute)

Arguments

See Also

H5Aopen()

Description

Creates an attribute, name, which is attached to the object specified by the identifier h5obj. The attribute name must be unique for the object.

Usage

H5Acreate(h5obj, name, dtype_id, h5space)

Arguments

Value

An object of class H5IdComponent representing a H5 attribute identifier.

Description

Delete an specified attribute of an HDF5 object

Usage

H5Adelete(h5obj, name)

Arguments

Description

Check whether an specific attribute exists for an HDF5 object

Usage

H5Aexists(h5obj, name)

Arguments

Description

Retrieves the name of the attribute specified by an HDF5 attribute object.

Usage

H5Aget_name(h5attribute)

Arguments

Value

A character vector of length 1 containing the name of the attribute.

Description

Get a copy of the attribute dataspace

Usage

H5Aget_space(h5attribute)

Arguments

Value

Returns an object of class H5IdComponent representing a H5 dataspace identifier

Description

Get a copy of the attribute datatype

Usage

H5Aget_type(h5attribute)

Arguments

Description

Open an attribute for an HDF5 object

Usage

H5Aopen(h5obj, name)

H5Aopen_by_name(h5obj, objname = ".", name)

H5Aopen_by_idx(
  h5obj,
  n,
  objname = ".",
  index_type = h5default("H5_INDEX"),
  order = h5default("H5_ITER")
)

Arguments

Value

An object of class H5IdComponent representing a H5 attribute identifier.

Description

Read data from an HDF5 attribute

Usage

H5Aread(h5attribute, buf = NULL, bit64conversion)

Arguments

Details

Internally, R does not support 64-bit integers. All integers in R are 32-bit integers. By setting bit64conversion='int', a coercing to 32-bit integers is enforced, with the risk of data loss, but with the insurance that numbers are represented as integers. bit64conversion='double' coerces the 64-bit integers to floating point numbers. doubles can represent integers with up to 54-bits, but they are not represented as integer values anymore. For larger numbers there is again a data loss. bit64conversion='bit64' is recommended way of coercing. It represents the 64-bit integers as objects of class 'integer64' as defined in the package 'bit64'. Make sure that you have installed 'bit64'. The datatype 'integer64' is not part of base R, but defined in an external package. This can produce unexpected behaviour when working with the data.

Value

If buf=NULL returns the contents of the attribute. Otherwise return 0 if attribute is read successfully.

Description

Write data to an HDF5 attribute

Usage

H5Awrite(h5attribute, buf)

Arguments

Description

This functions can be used in two ways. Firstly, it can be passed one or more H5IdComponent objects and it'll will try to close all of them regardless of the whether they represent a file, group, dataset etc. This can be easier than making multiple calls to H5Fclose(), H5Gclose(), etc.

Usage

h5closeAll(...)

Arguments

Details

Secondly, cccasionally references to HDF5 files, groups, datasets etc can be created and not closed correctly. Maybe because a function stopped before getting to the close statement, or the open handle was not assigned to an R variable. If no arguments are provide this function identifies all open handles and closes them.

Value

Doesn't return anything. Called for the side-effect of closing open HDF5 handles.

Author(s)

Mike Smith

Examples

## create an empty file and then re-open it
h5File <- tempfile(pattern = "ex_h5closeAll.h5")
h5createFile(h5File)
H5Fopen(h5File)

## list all open identifiers
h5listIdentifier()

## close all open identifiers and verify
h5closeAll()
h5listIdentifier()

Description

Access to HDF5 constants.

Usage

h5const(type = "")

h5constType()

h5default(type = "")

Arguments

Details

These functions provide a list of HDF5 constants that are defined in the R package. h5constType provides a list of group names and h5const gives the constants defined within a group. h5default gives the default choice for each group.

Value

A character vector with names of HDF5 constants or groups.

Author(s)

Bernd Fischer

Examples

h5constType()[1]
h5const(h5constType()[1])

Description

Additional functions for finding details of dataset chunking.

Usage

H5Dchunk_dims(h5dataset)

H5Dis_chunked(h5dataset)

Arguments

Details

These functions do not map directly to the HDF5 C API but follow the same style and are included as potentially useful additions.

• H5Dis_chunked tests whether a dataset is chunked.

• H5Dchunk_dims will return the dimensions of the dataset chunks.

Value

• H5Dchunk_dims: If the supplied dataset is chunked returns a vector, with length equal to the rank of the dataset, containing the size of the dataset dimensions. Returns NULL if the given dataset is not chunked.

• H5Dis_chunked: returns TRUE if a dataset is chunked and FALSE otherwise.

Author(s)

Mike Smith

Description

Close an open HDF5 dataset

Usage

H5Dclose(h5dataset)

Arguments

Description

Create a new HDF5 dataset

Usage

H5Dcreate(
  h5loc,
  name,
  dtype_id,
  h5space,
  lcpl = NULL,
  dcpl = NULL,
  dapl = NULL
)

Arguments

Value

An object of class H5IdComponent representing the opened dataset.

Description

Return a copy of the dataset creation property list for a dataset

Usage

H5Dget_create_plist(h5dataset)

Arguments

Description

Return a copy of the HDF5 dataspace for a dataset

Usage

H5Dget_space(h5dataset)

Arguments

Value

Returns an object of class H5IdComponent representing a HDF5 dataspace identifier

Description

H5Dget_storage_size returns the amount of storage, in bytes, allocated in an HDF5 file to hold a given dataset. This is the amount of space required on-disk, which not typically a good indicator of the amount of memory that will be required to read the complete dataset.

Usage

H5Dget_storage_size(h5dataset)

Arguments

Value

Returns an integer giving the number of bytes allocated in the file to the dataset.

Description

Return a copy of the HDF5 datatype for a dataset

Usage

H5Dget_type(h5dataset)

Arguments

Description

Open an existing HDF5 dataset

Usage

H5Dopen(h5loc, name, dapl = NULL)

Arguments

Value

An object of class H5IdComponent representing the opened dataset. To prevent memory leaks this must be closed with a call to H5Dclose() when no longer needed.

Examples

h5file <- tempfile(fileext = ".h5")
h5createFile( h5file )
h5createDataset( h5file, dataset = "A", dims = 10)

fid <- H5Fopen( h5file )
did <- H5Dopen( h5loc = fid, name = "A")
did

## rember to close open handles
H5Dclose( did )
H5Fclose( fid )

Description

H5Dread() reads a (partial) dataset from an HDF5 file into the R session.

Usage

H5Dread(
  h5dataset,
  h5spaceFile = NULL,
  h5spaceMem = NULL,
  buf = NULL,
  compoundAsDataFrame = TRUE,
  bit64conversion,
  drop = FALSE
)

Arguments

Details

Internally, R does not support 64-bit integers. All integers in R are 32-bit integers. By setting bit64conversion='int', a coercing to 32-bit integers is enforced, with the risk of data loss, but with the insurance that numbers are represented as integers. bit64conversion='double' coerces the 64-bit integers to floating point numbers. doubles can represent integers with up to 54-bits, but they are not represented as integer values anymore. For larger numbers there is again a data loss. bit64conversion='bit64' is recommended way of coercing. It represents the 64-bit integers as objects of class 'integer64' as defined in the package 'bit64'. Make sure that you have installed 'bit64'. The datatype 'integer64' is not part of base R, but defined in an external package. This can produce unexpected behaviour when working with the data.

Description

Change the dimensions of an HDF5 dataset

Usage

H5Dset_extent(h5dataset, size)

Arguments

Details

This function can only be applied to datasets that meet the following criteria:

• A chunked dataset with unlimited dimensions

• A chunked dataset with fixed dimensions if the new dimension sizes are less than the maximum sizes set with maxdims

Value

A logical vector of length 1. Value will be TRUE if the operation was sucessful and FALSE otherwise.

Author(s)

Bernd Fischer, Mike Smith

Description

Write data to dataset

Usage

H5Dwrite(h5dataset, buf, h5type = NULL, h5spaceMem = NULL, h5spaceFile = NULL)

Arguments

Description

Close access to an HDF5 file

Usage

H5Fclose(h5file)

Arguments

Description

Create an HDF5 file

Usage

H5Fcreate(
  name,
  flags = h5default("H5F_ACC"),
  fcpl = NULL,
  fapl = NULL,
  native = FALSE
)

Arguments

Description

Flush all buffers associated with a file to disk

Usage

H5Fflush(h5file, scope = h5default("H5F_SCOPE"))

Arguments

Description

H5Fget_filesize() returns the size in bytes of the HDF5 file specified by h5file.

Usage

H5Fget_filesize(h5file)

Arguments

Description

Retrieve the name of the file to which an object belongs

Usage

H5Fget_name(h5obj)

Arguments

Examples

## use an example file and show its location
h5file <- system.file("testfiles", "h5ex_t_array.h5", package = "rhdf5")
h5file

## open a file handle and confirm we can identify the file it points to
fid <- H5Fopen(h5file)
H5Fget_name(fid)

## H5Fget_name() can be applied to group and dataset handles too
gid <- H5Gopen(fid, name = "/")
did <- H5Dopen(fid, name = "DS1")
H5Fget_name(gid)
H5Fget_name(did)

## tidy up
H5Dclose(did)
H5Gclose(gid)
H5Fclose(fid)

Description

Get property lists associated with an HDF5 file

Usage

H5Fget_create_plist(h5file)

H5Fget_access_plist(h5file)

Arguments

Description

H5Fis_hdf5() determines whether a file is in the HDF5 format.

Usage

H5Fis_hdf5(name, showWarnings = TRUE)

Arguments

Value

Returns TRUE, if the file is an HDF5 file, or FALSE otherwise. In the case the file doesn't exist, NA is returned

Description

Open an existing HDF5 file

Usage

H5Fopen(name, flags = h5default("H5F_ACC_RD"), fapl = NULL, native = FALSE)

Arguments

Details

Possible values for the flags argument are H5F_ACC_RDWR and H5F_ACC_RDONLY. Note that HDF5's "Single Write Multiple Reader (SWMR) mode is not currently supported via rhdf5.

Description

These low level functions provide general library functions for HDF5.

Usage

H5open()

H5close()

H5garbage_collect()

H5get_libversion()

Value

• H5open initializes the HDF5 library.

• H5close flushes all data to disk, closes all open identifiers, and cleans up memory.

• H5garbage_collect cleans up memory.

• H5get_libversion returns the version number of the HDF5 C-library.

Author(s)

Bernd Fischer, Mike Smith

Examples

## Not run: 
  H5open()
  H5close()
  H5garbage_collect()
  H5get_libversion()

## End(Not run)

Description

Close a specified group

Usage

H5Gclose(h5group)

Arguments

Description

H5Gcreate is used to a new group and link it into a file.

Usage

H5Gcreate(h5loc, name)

Arguments

Description

Create a new HDF5 group without linking it into a file

Usage

H5Gcreate_anon(h5loc)

Arguments

Value

H5Gcreate_anon returns an object of class H5IdComponent representing the newly created group. However at this point is is still anonymous, and must be linked into the file structure via H5Olink(). If this is not done, the group will be deleted from the file when it is closed.

See Also

H5Gcreate(), H5Olink()

Description

Retrieve information about a group

Usage

H5Gget_info(h5loc)

H5Gget_info_by_name(h5loc, group_name)

H5Gget_info_by_idx(
  h5loc,
  n,
  group_name = ".",
  index_type = h5default("H5_INDEX"),
  order = h5default("H5_ITER")
)

Arguments

Value

A list with group information

Examples

h5file <- system.file("testfiles", "multiple_dtypes.h5", package="rhdf5")
fid <- H5Fopen(h5file)
gid <- H5Gopen(fid, "/foo")
gid
H5Gget_info(gid)
H5Gclose(gid)

## the "get_info_by" functions take the H5 object that contains the
## group(s) of interest.  We can retrieve information by index or by name
H5Gget_info_by_idx(fid, 3)
H5Gget_info_by_name(fid,"/foo")

H5Fclose(fid)

Description

Open a specified group

Usage

H5Gopen(h5loc, name)

Arguments

Value

An object of class H5IdComponent representing the opened group. When access to the group is no longer needed this should be released with H5Gclose() to prevent resource leakage.

See Also

H5Gclose()

Description

A class representing a HDF5 identifier handle. HDF5 identifiers represent open files, groups, datasets, dataspaces, attributes, and datatypes.

Usage

## S4 method for signature 'H5IdComponent'
show(object)

## S4 method for signature 'H5IdComponent,character'
e1 & e2

## S4 method for signature 'H5IdComponent'
x$name

## S4 replacement method for signature 'H5IdComponent'
x$name <- value

## S4 method for signature 'H5IdComponent'
x[i, j, ..., drop = TRUE]

## S4 replacement method for signature 'H5IdComponent'
x[i, j, ...] <- value

Arguments

Methods (by generic)

• show(H5IdComponent): Print details of the object to screen.

• e1 & e2: Returns a group handle or dataset handle for the group or dataset name in the HDF5 location h5loc. h5loc can either be a file handle as returned by H5Fopen or a group handle as e.g. returned by h5f$g1 or h5f$'/g1/g2'.

• $: Reads the HDF5 object name in the HDF5 location x. x can either be a file handle as returned by H5Fopen or a group handle as e.g. returned by h5f$g1 or h5f$'/g1/g2'.

• `$`(H5IdComponent) <- value: Writes the assigned object to to the HDF5 file at location e1. e1 can either be a file handle as returned by H5Fopen or a group handle as e.g. returned by h5f$g1 or h5f$'/g1/g2's. The storage.mode of the assigned object has to be compatible to the datatype of the HDF5 dataset. The dimension of the assigned object have to be identical the dimensions of the HDF5 dataset. To create a new HDF5 dataset with specific properties (e.g. compression level or chunk size), please use the function h5createDataset first.

• [: Subsetting of an HDF5 dataset. The function reads a subset of an HDF5 dataset. The given dimensions have to fit the dimensions of the HDF5 dataset.

• `[`(H5IdComponent) <- value: Subsetting of an HDF5 dataset. The function writes an R data object to a subset of an HDF5 dataset. The given dimensions have to fit the dimensions of the HDF5 dataset. The HDF5 dataset has to be created beforehand, e.g. by h5createDataset.

Slots

ID

integer of length 1. Contains the handle of C-type hid_t.

native

An object of class logical. If TRUE, array-like objects are treated as stored in HDF5 row-major rather than R column-major orientation. Using native = TRUE increases HDF5 file portability between programming languages. A file written with native = TRUE should also be read with native = TRUE

Description

Retrieve the name of an object from a given identifier

Usage

H5Iget_name(h5obj)

Arguments

Description

Possible types returned by the function are:

• H5I_FILE

• H5I_GROUP

• H5I_DATATYPE

• H5I_DATASPACE

• H5I_DATASET

• H5I_ATTR

Usage

H5Iget_type(h5identifier)

Arguments

Value

Returns a character vector of length 1 containing the HDF5 type for the supplied identifier.

Examples

h5file <- system.file("testfiles", "h5ex_t_array.h5", package="rhdf5")
fid <- H5Fopen(h5file)
gid <- H5Gopen(fid, "/")

## identify the HDF5 types for these identifiers
H5Iget_type(fid)
H5Iget_type(gid)

## tidy up
H5Gclose(gid)
H5Fclose(fid)

Description

An identifier is no longer valid after it has been closed.

Usage

H5Iis_valid(h5identifier)

Arguments

Value

A logical of length 1. TRUE is the identifier is valid, FALSE if not.

Examples

h5file <- system.file("testfiles", "h5ex_t_array.h5", package="rhdf5")
fid <- H5Fopen(h5file)

## test whether the identifer to the opened file is valid
H5Iis_valid(fid)

## the file ID is no longer valid after it has been closed
H5Fclose(fid)
H5Iis_valid(fid)

Description

Copy a link from one location to another

Usage

H5Lcopy(h5loc, name, h5loc_dest, name_dest, lcpl = NULL, lapl = NULL)

Arguments

Description

H5Lcreate_external() creates a new external link. An external link is a soft link to an object in a different HDF5 file from the location of the link.

Usage

H5Lcreate_external(target_file_name, target_obj_name, link_loc, link_name)

Arguments

Examples

## The example below creates a new HDF5 file in a temporary director, and then
## links to the group "/foo" found in the file "multiple_dtypes.h5" 
## distributed with the package.

h5File1 <- system.file("testfiles", "multiple_dtypes.h5", package="rhdf5")
h5File2 <- tempfile(pattern = "H5L_2_", fileext = ".h5")
h5createFile(h5File2)

## open the new file & create a link to the group "/foo" in the original file
fid <- H5Fopen(h5File2)
H5Lcreate_external(target_file_name = h5File1, target_obj_name = "/foo", 
  link_loc = fid, link_name = "/external_link")
H5Fclose(fid)

## check the new file has a group called "/external_link"
h5ls(h5File2)

Description

Remove a link from a group

Usage

H5Ldelete(h5loc, name)

Arguments

Examples

h5file <- tempfile(pattern = "_ex_H5L.h5")

# create an hdf5 file and a group
h5createFile( h5file )
h5createGroup(h5file,"/foo")

# reopen file and confirm "/foo" exists but "/baa" does not
fid <- H5Fopen(h5file)
H5Lexists(fid, "/foo")

# remove the link to "/foo" and confirm it no longer exists
H5Ldelete(fid, "/foo")
H5Lexists(fid, "/foo")

H5Fclose(fid)

Description

Confirm existence of a link

Usage

H5Lexists(h5loc, name)

Arguments

Description

H5Lget_info() identifies the type of link specified by the the h5loc and name arguments. This is more limited than the equivalent function in the standard HDF5 library.

Usage

H5Lget_info(h5loc, name)

Arguments

Value

A character vector of length 1 giving the type of link. Possible values are: H5L_TYPE_HARD, H5L_TYPE_SOFT, H5L_TYPE_EXTERNAL, H5L_TYPE_ERROR

Description

A list of all valid HDF5 identifiers. H5 objects should be closed after usage to release resources.

Usage

h5listIdentifier()

h5validObjects(native = FALSE)

Arguments

Value

h5validObjects returns a list of H5IdComponent objects. h5listIdentifier prints the valid identifiers on screen and returns NULL.

Author(s)

Bernd Fischer, Mike Smith

Examples

h5File <- tempfile("ex_list_identifier.h5")

h5createFile(h5File)

# create groups
h5createGroup(h5File,"foo")

h5listIdentifier()
h5validObjects()

Description

Move a link within an HDF5 file

Usage

H5Lmove(h5loc, name, h5loc_dest, name_dest, lcpl = NULL, lapl = NULL)

Arguments

Examples

## create an HDF5 file with a single group
## that contains a dataset of 10 numbers
h5file <- tempfile(fileext = ".h5")
h5createFile(h5file)
h5createGroup(h5file, "/foo")
h5write(1:10, h5file, name = "/foo/vector1")
## check the structure is what we expect
h5ls(h5file)

## open the file, the group where the dataset currently is
## and the root group
fid <- H5Fopen(name = h5file)
gid1 <- H5Gopen(fid, "/foo")
gid2 <- H5Gopen(fid, "/")
## move the dataset to the root of the file and rename it
H5Lmove(gid1, "vector1", gid2, "vector_new")
h5closeAll()
## check the dataset has moved out of the foo group
h5ls(h5file)

## we can also provide the ID of the HDF5 file
## and use the "name" arguments to move between groups
fid <- H5Fopen(name = h5file)
H5Lmove(fid, "/vector_new", fid, "/foo/vector_newer")
H5Fclose(fid)
h5ls(h5file)

Description

List the content of an HDF5 file.

Usage

h5ls(
  file,
  recursive = TRUE,
  all = FALSE,
  datasetinfo = TRUE,
  index_type = h5default("H5_INDEX"),
  order = h5default("H5_ITER"),
  s3 = FALSE,
  s3credentials = NULL,
  native = FALSE
)

Arguments

Value

h5ls returns a data.frame with the file content.

Author(s)

Bernd Fischer, Mike L. Smith

References

https://portal.hdfgroup.org/display/HDF5

See Also

h5dump()

Examples

h5File <- tempfile(pattern = "ex_dump.h5")
h5createFile(h5File)

# create groups
h5createGroup(h5File,"foo")
h5createGroup(h5File,"foo/foobaa")

# write a matrix
B = array(seq(0.1,2.0,by=0.1),dim=c(5,2,2))
attr(B, "scale") <- "liter"
h5write(B, h5File,"foo/B")

# list content of hdf5 file
h5ls(h5File,all=TRUE)

# list content of an hdf5 file in a public S3 bucket

h5ls(file = "https://rhdf5-public.s3.eu-central-1.amazonaws.com/h5ex_t_array.h5", s3 = TRUE)

Description

Close an HDF5 object

Usage

H5Oclose(h5obj)

Arguments

See Also

H5Oopen()

Description

Copies an HDF5 object

Usage

H5Ocopy(h5loc, name, h5loc_dest, name_dest, obj_cpy_pl = NULL, lcpl = NULL)

Arguments

Examples

## Create a temporary copy of an example file check the contents
example_file <- system.file("testfiles", "h5ex_t_array.h5", package="rhdf5")
file.copy(example_file, tempdir())
h5_file <- file.path(tempdir(), "h5ex_t_array.h5")
h5ls(h5_file)

## open the example file and create a new, empty, file
fid1 <- H5Fopen( h5_file )
h5_file2 <- tempfile(fileext = ".h5")
fid2 <- H5Fcreate( h5_file2 )

## We can copy a dataset inside the same file
H5Ocopy(h5loc = fid1, name = "DS1", h5loc_dest = fid1, name_dest = "DS2")
## Or to a different file
H5Ocopy(h5loc = fid1, name = "DS1", h5loc_dest = fid2, name_dest = "DS1_copy")

## if we want to create a new group hierarchy we can use a link creation property list
lcpl <- H5Pcreate("H5P_LINK_CREATE")
H5Pset_create_intermediate_group( lcpl, create_groups = TRUE )
H5Ocopy(h5loc = fid1, name = "DS1", h5loc_dest = fid2, 
        name_dest = "/foo/baa/DS1_nested", lcpl = lcpl)

## tidy up
H5Pclose(lcpl)
H5Fclose(fid1)
H5Fclose(fid2)

## Check we now have groups DS1 and DS2 in the original file
h5ls( h5_file )
## Check we have a copy of DS1 at the root and nests in the new file
h5ls( h5_file2 )

Description

Find the number of attributes associated with an HDF5 object

Usage

H5Oget_num_attrs(h5obj)

H5Oget_num_attrs_by_name(h5loc, name)

Arguments

Details

These functions are not part of the standard HDF5 C API.

Value

Returns a vector of length 1 containing the number of attributes the specified object has.

Description

Create a hard link to an object in an HDF5 file

Usage

H5Olink(h5obj, h5loc, newLinkName, lcpl = NULL, lapl = NULL)

Arguments

See Also

H5Gcreate_anon

Examples

## Create a temporary copy of an example file, and open it
example_file <- system.file("testfiles", "h5ex_t_array.h5", package="rhdf5")
file.copy(example_file, tempdir())
h5_file <- file.path(tempdir(), "h5ex_t_array.h5")
fid <- H5Fopen( h5_file )

## create a new group without a location in the file
gid <- H5Gcreate_anon(fid)

## create link to newly create group
## relative to the file identifier
H5Olink(h5obj = gid, h5loc = fid, newLinkName = "foo")

## tidy up
H5Gclose(gid)
H5Fclose(fid)

## Check we now have a "/foo" group
h5ls( h5_file )

Description

Open an object in an HDF5 file

Usage

H5Oopen(h5loc, name)

Arguments

Value

An object of class H5IdComponent if the open operation was successful. FALSE otherwise.

See Also

H5Oclose()

Examples

h5File <- tempfile(pattern = "ex_H5O.h5")

# create an hdf5 file and write something
h5createFile(h5File)
h5createGroup(h5File,"foo")
B = array(seq(0.1,2.0,by=0.1),dim=c(5,2,2))
h5write(B, h5File,"foo/B")

# reopen file and dataset and get object info
fid <- H5Fopen(h5File)
oid = H5Oopen(fid, "foo")
H5Oget_num_attrs(oid)
H5Oclose(oid)
H5Fclose(fid)

Description

Get and set the size of the chunks used to store a chunked layout dataset

Usage

H5Pset_chunk(h5plist, dim)

H5Pget_chunk(h5plist)

Arguments

Details

Note that a necessary side effect of running this function is that the layout of the dataset will be changes to H5D_CHUNKED if it is not already set to this.

See Also

H5Pset_layout()

Description

Set parameters for the raw data chunk cache

Usage

H5Pset_chunk_cache(h5plist, rdcc_nslots, rdcc_nbytes, rdcc_w0)

Arguments

Description

Get and set whether to create missing intermediate groups

Usage

H5Pset_create_intermediate_group(h5plist, create_groups = TRUE)

H5Pget_create_intermediate_group(h5plist)

Arguments

Examples

pid <- H5Pcreate("H5P_LINK_CREATE")

## by default intermediate groups are not created
H5Pget_create_intermediate_group( pid )

## Change the setting so groups will be created

H5Pget_create_intermediate_group( pid )

## tidy up
H5Pclose(pid)

Description

Set the time when fill values are written to a dataset

Usage

H5Pset_fill_time(h5plist, fill_time = h5default("H5D_FILL_TIME"))

H5Pget_fill_time(h5plist)

Arguments

Description

H5Pset_fill_value sets the fill value for a dataset in the dataset creation property list.

Usage

H5Pset_fill_value(h5plist, value)

Arguments

See Also

H5P_fill_time,H5Pfill_value_defined

Description

Possible options for the layout argument are:

• H5D_COMPACT

• H5D_CONTIGUOUS

• H5D_CHUNKED

• H5D_VIRTUAL

Usage

H5Pset_layout(h5plist, layout = h5default("H5D"))

H5Pget_layout(h5plist)

Arguments

Details

The names of the layout types can also be obtained via h5const("H5D").

Description

Control the range of HDF5 library versions that will be compatible with a file.

Usage

H5Pset_libver_bounds(
  h5plist,
  libver_low = "H5F_LIBVER_EARLIEST",
  libver_high = "H5F_LIBVER_LATEST"
)

H5Pget_libver_bounds(h5plist)

Arguments

Description

Return information about the filter pipeline applied to a dataset creation property list.

Usage

H5Pall_filters_avail(h5plist)

H5Pget_nfilters(h5plist)

H5Pget_filter(h5plist, idx)

Arguments

Details

• H5Pall_filters_avail() checks whether all filters required to process a dataset are available to rhdf5. This can be required if reading files created with other HDF5 software.

• H5Pget_nfilters() returns the number of filters in the dataset chunk processing pipeline.

• H5Pget_filter() provides details of a specific filter in the pipeline. This includes the filter name and the parameters provided to it e.g. compression level.

Description

H5Pclose() terminates access to a property list. All property lists should be closed when they no longer need to be accessed. This frees resources used by the property list. Failing to call H5Pclose() can lead to memory leakage over time.

Usage

H5Pclose(h5plist)

Arguments

Description

Copy an existing property list to create a new property list

Usage

H5Pcopy(h5plist)

Arguments

Description

Create a new HDF5 property list

Usage

H5Pcreate(type = h5default("H5P"), native = FALSE)

Arguments

Description

Determine whether a property list has a fill value defined

Usage

H5Pfill_value_defined(h5plist)

Arguments

Details

Note that the return value for this function is slightly different from the C version. The C API provides three return types and can, in the case that a fill value is defined, differentiate whether the value is the HDF5 library default or has been set by the application.

Value

TRUE if the fill value is defined, FALSE if not. Will return NULL if there is a problem determining the status of the fill value.

Description

Return the property list class identifier for a property list

Usage

H5Pget_class(h5plist)

Arguments

Description

Get version information for objects in a file creation property list

Usage

H5Pget_version(h5plist)

Arguments

Value

Named integer vector

Description

Set whether to record timestamps for operations performed on an HDF5 object.

Usage

H5Pset_obj_track_times(h5plist, track_times = TRUE)

H5Pget_obj_track_times(h5plist)

Arguments

Details

Objects created using high-level rhdf5 functions like h5createDataset() will have this setting turned off. This was done to ensure otherwise identical files returned the same md5 hash. This differs from the default setting in HDF5, which is for objects to record the times operations were performed on them.

Description

Add the BLOSC filter to the chunk processing pipeline.

Usage

H5Pset_blosc(h5plist, h5tid, method = 1L, level = 6L, shuffle = TRUE)

Arguments

Description

Add the BZIP2 filter to the chunk processing pipeline.

Usage

H5Pset_bzip2(h5plist, level = 2L)

Arguments

Description

Valid values for the compression level range from 0 (no compression) to 9 (best compression, slowest speed). Note that applying this function with level = 0 does not mean the filter is removed. It is still part of the filter pipeline, but no compression is performed. The filter will still need to be available on any system that reads a file created with this setting

Usage

H5Pset_deflate(h5plist, level)

Arguments

Description

The read-only S3 virtual file driver can be used to read files hosted remotely on Amazon's S3 storage.

Usage

H5Pset_fapl_ros3(h5plist, s3credentials = NULL)

Arguments

Details

To access files in a private Amazon S3 bucket you will need to provide three additional details: The AWS region where the files are hosted, your AWS access key ID, and your AWS secret access key. More information on how to obtain AWS access keys can be found at https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys. These are provided as a list to the s3credentials argument. If you are accessing public data this argument should be NULL.

Examples

## this doesn't work on the Bioconductor Mac build machine
## Not run: 
pid <- H5Pcreate("H5P_FILE_ACCESS")
H5Pset_fapl_ros3( pid )
H5Pclose(pid)

## End(Not run)

Description

Add a filter to the dataset filter pipeline.

Usage

H5Pset_filter(h5plist, filter_id, is_mandatory = FALSE, cd_values)

Arguments

Description

Get and set the 1/2 rank of an indexed storage B-tree

Usage

H5Pset_istore_k(h5plist, ik)

H5Pget_istore_k(h5plist)

Arguments

Description

Add the LZF filter to the chunk processing pipeline.

Usage

H5Pset_lzf(h5plist, h5tid)

Arguments

Description

Add the N-Bit filter to the chunk processing pipeline.

Usage

H5Pset_nbit(h5plist)

Arguments

Value

Returns (invisibly) an integer vector of length 1. The only element of this vector will be non-negative if the filter was set successfully and negative otherwise.

Description

Get and set shared object header message index properties

Usage

H5Pset_shared_mesg_index(
  h5plist,
  index_num,
  mesg_type_flags = h5default(type = "H5O_SHMESG_FLAG"),
  min_mesg_size
)

H5Pget_shared_mesg_index(h5plist, index_num)

Arguments

Value

H5Pget_shared_mesg_index() returns a list of length 2. The first element is the types of messages that may be stored in the index, the second element is the minimum message size.

Description

Get and set the number of object header message indexes

Usage

H5Pset_shared_mesg_nindexes(h5plist, nindexes)

H5Pget_shared_mesg_nindexes(h5plist)

Arguments

Description

Get and set threshold values for storage of shared object header message indexes

Usage

H5Pset_shared_mesg_phase_change(h5plist, max_list, min_btree)

H5Pget_shared_mesg_phase_change(h5plist)

Arguments

Description

Add the shuffle filter to the chunk processing pipeline.

Usage

H5Pset_shuffle(h5plist)

Arguments

Value

Returns (invisibly) an integer vector of length 1. The only element of this vector will be non-negative if the filter was set successfully and negative otherwise.

Description

Get and set the sizes of offsets and lengths used in an HDF5 file

Usage

H5Pset_sizes(h5plist, sizeof_addr, sizeof_size)

H5Pget_sizes(h5plist)

Arguments

Description

Get and set the size of the symbol table B-tree 1/2 rank and the leaf node 1/2 size

Usage

H5Pset_sym_k(h5plist, ik, lk)

H5Pget_sym_k(h5plist)

Arguments

Description

Add the SZIP compression filter to the chunk processing pipeline.

Usage

H5Pset_szip(h5plist, options_mask, pixels_per_block)

Arguments

References

https://portal.hdfgroup.org/display/HDF5/Szip+Compression+in+HDF+Products

Description

Get and set the user block size

Usage

H5Pset_userblock(h5plist, size)

H5Pget_userblock(h5plist)

Arguments

Description

The H5R functions can be used for creating or working with references to specific objects and data regions in an HDF5 file.

Author(s)

Mike Smith

Examples

library(rhdf5)

## first we'll create a file with a group named "foo" and a
## 1-dimensional dataset named "baa" inside that group.
file_name <- tempfile(fileext = ".h5")
h5createFile(file_name)
h5createGroup(file = file_name, group = "/foo")
h5write(1:100, file=file_name, name="/foo/baa")


fid <- H5Fopen(file_name)
ref_to_group <- H5Rcreate(fid, name = "/foo")
ref_to_dataset <- H5Rcreate(fid, name = "/foo/baa")
two_refs <- c(ref_to_group, ref_to_dataset)
two_refs

## the size of this dataspace is the number of object references
## we want to store
sid <- H5Screate_simple(2)
tid <- H5Tcopy(dtype_id = "H5T_STD_REF_OBJ")
did <- H5Dcreate(fid, name = "object_refs", dtype_id = tid, h5space = sid)
H5Dwrite(did, two_refs)
H5Dclose(did)
H5Sclose(sid)
H5Fclose(fid)

Description

Creates a reference to an object or dataset selection inside an HDF5 file.

Usage

H5Rcreate(h5loc, name, ref_type = "H5R_OBJECT", h5space = NULL)

Arguments

Value

An H5Ref object storing the reference.

Description

Given a reference and the file to which that reference applies, H5Rdeference() will open the reference object and return an identifier.

Usage

H5Rdereference(ref, h5loc)

Arguments

Details

If ref contains more than one reference, only the first reference will be used. It must be subset with [ if one of the other stored references should be opened.

Value

An object of class H5IdComponent representing the opened object referenced by ref. This should be closed with the appropriate function e.g. H5Dclose(), H5Oclose(), etc. when no longer needed.

Description

A class representing one or more HDF5 references.

Usage

## S4 method for signature 'H5Ref'
show(object)

## S4 method for signature 'H5Ref'
length(x)

## S4 method for signature 'H5Ref'
c(x, ...)

## S4 method for signature 'H5Ref'
x[i]

Arguments

Details

The length of the val slot is dependent on both the number and type of references stored in the object. H5R_OBJECT references are stored in 8 bytes, while H5R_DATASET_REGION references require 12 bytes. The length of val will then be a multiple of 8 or 12 respectively. This also means that references of different types cannot be combined in a single object.

Methods (by generic)

• show(H5Ref): Print details of the object to screen.

• length(H5Ref): Return the number of references stored in an H5Ref object.

• c(H5Ref): Combine two or more H5Ref objects. Objects must all contain the same type of reference, either H5R_OBJECT or H5R_DATASET_REFERENCE.

• [: Subset an H5Ref object.

Slots

val

raw vector containing the byte-level representation of each reference.

type

integer of length 1, which maps to either H5R_OBJECT or H5R_DATASET_REGION.

Description

Return the name of the object that a reference points to

Usage

H5Rget_name(ref, h5loc)

Arguments

Value

Character string of length 1 giving the name of the referenced object.

Description

Identify the type of object that a reference points to

Usage

H5Rget_obj_type(ref, h5loc)

Arguments

Value

Character string of length 1 identifying the object type. Valid return values are: "GROUP", "DATASET", and "NAMED_DATATYPE".

Description

Given a dataset region reference, this function will return the dataspace and selection required to read the data points indicated by the reference.

Usage

H5Rget_region(ref, h5loc)

Arguments

Value

An object of class H5IdComponent representing the dataspace of the dataset that ref points to. The dataspace will have the selection set that matches the selection pointed to by ref. This should be closed using H5Sclose() when no longer required.

Description

Close and release a dataspace

Usage

H5Sclose(h5space)

Arguments

See Also

H5Screate()

Description

Combines a hyperslab selection specified by start, stride, count and block arguments with the current selection for the dataspace represented by h5space.

Usage

H5Scombine_hyperslab(
  h5space,
  op = h5default("H5S_SELECT"),
  start = NULL,
  stride = NULL,
  count = NULL,
  block = NULL
)

Arguments

Value

An H5IdComponent object representing a new dataspace with the generated selection.

See Also

H5Scombine_select(), H5Sselect_hyperslab()

Examples

## create a 1 dimensional dataspace
sid_1 <- H5Screate_simple(dims = 20)

## select a single block of 5 points in sid_1
## this is equivalent to [11:16] in R syntax
H5Sselect_hyperslab(sid_1, start = 11, stride = 1, 
                    block = 5, count = 1)#

## combine the existing selection with a new
## selection consisting of 2 blocks each of 1 point
## equivalent to [c(3,5)] in R syntax
sid_2 <- H5Scombine_hyperslab(sid_1, op = "H5S_SELECT_OR",
                              start = 3, stride = 2, 
                              block = 1, count = 2)

## confirm we have selected 5 in our original dataspace
## and 7 points in the newly created dataspace
H5Sget_select_npoints(sid_1)
H5Sget_select_npoints(sid_2)

## tidy up
H5Sclose(sid_1)
H5Sclose(sid_2)

Description

Combine two selections

Usage

H5Scombine_select(h5space1, op = h5default("H5S_SELECT"), h5space2)

Arguments

Value

Returns an H5IdComponent object representing a new dataspace. The new dataspace will have the same extent as h5space1 with the hyperslab selection being the result of combining the selections of h5space1 and h5space2.

See Also

H5Scombine_hyperslab()

Examples

## create two 1 dimensional dataspaces
## of different sizes
sid_1 <- H5Screate_simple(dims = 20)
sid_2 <- H5Screate_simple(dims = 10)

## select a single block of 5 points in sid_1
## this is equivalent to [11:16] in R syntax
H5Sselect_hyperslab(sid_1, start = 11, stride = 1, 
                    block = 5, count = 1)

## select 2 blocks of 1 point from sid_2
## equivalent to [c(3,5)] in R syntax
H5Sselect_hyperslab(sid_2, start = 3, stride = 2, 
                    block = 1, count = 2)

## confirm we have select 5 and 2 points resepectively
H5Sget_select_npoints(sid_1)
H5Sget_select_npoints(sid_2)

## combine the two dataset selections keeping points that
## are in one or both of the selections
sid_3 <- H5Scombine_select(sid_1, "H5S_SELECT_OR", sid_2)

## extent of the new dataset is the same as sid_1
sid_3
## confirm the selection contains 7 points
H5Sget_select_npoints(sid_3)

## tidy up
H5Sclose(sid_1)
H5Sclose(sid_2)
H5Sclose(sid_3)

Description

H5S_copy() creates an exact copy of a given dataspace.

Usage

H5Scopy(h5space)

Arguments

Value

If the copying is successful returns an object of class H5IdComponent representing the new dataspace. Otherwise returns FALSE.

Description

Create a new dataspace of a specified type

Usage

H5Screate(type = h5default("H5S"), native = FALSE)

Arguments

Value

Returns an object of class H5IdComponent representing a dataspace.

See Also

H5Screate_simple

Description

Create a simple dataspace

Usage

H5Screate_simple(dims, maxdims, native = FALSE)

Arguments

Value

Returns an object of class H5IdComponent representing a dataspace.

See Also

H5Screate

Description

Find the number of elements in a dataspace selection

Usage

H5Sget_select_npoints(h5space)

Arguments

Description

Find the size of a dataspace

Usage

H5Sget_simple_extent_dims(h5space)

Arguments

Description

In HDF5 a dataspace is considered "simple" if it represents a regular N-dimensional array of points. Currently (HDF 1.10.7) all dataspaces are simple. Support for complex dataspaces is planned for future HDF versions.

Usage

H5Sis_simple(h5space)

Arguments

Description

Set the selection region of a dataspace to include all elements

Usage

H5Sselect_all(h5space)

Arguments

Description

Combines a hyperslab selection specified by start, stride, count and block arguments with the current selection for the dataspace represented by h5space.

Usage

H5Sselect_hyperslab(
  h5space,
  op = h5default("H5S_SELECT"),
  start = NULL,
  stride = NULL,
  count = NULL,
  block = NULL
)

Arguments

Details

H5Sselect_hyperslab is similar to, but subtly different from, H5Scombine_hyperslab(). The former modifies the selection of the dataspace provided in the h5space argument, while the later returns a new dataspace with the combined selection.

Examples

## create a 1 dimensional dataspace
sid_1 <- H5Screate_simple(dims = 20)

## select a single block of 5 points in sid_1
## this is equivalent to [11:16] in R syntax
H5Sselect_hyperslab(sid_1, start = 11, stride = 1, 
                    block = 5, count = 1)
                    
## confirm we have selected 5 in our original dataspace
H5Sget_select_npoints(sid_1)

## combine the existing selection with a new
## selection consisting of 2 blocks each of 1 point
## equivalent to [c(3,5)] in R syntax
H5Sselect_hyperslab(sid_1, op = "H5S_SELECT_OR",
                     start = 3, stride = 2, 
                     block = 1, count = 2)

## The dataspace now has 7 points selected
H5Sget_select_npoints(sid_1)

## tidy up
H5Sclose(sid_1)

Description

Combines a hyperslab selection specified by start, stride, count and block arguments with the current selection for the dataspace represented by h5space.

Usage

H5Sselect_index(h5space, index)

Arguments

Details

H5Sselect_hyperslab is similar to, but subtly different from, H5Scombine_hyperslab(). The former modifies the selection of the dataspace provided in the h5space argument, while the later returns a new dataspace with the combined selection.

Examples

## create a 1 dimensional dataspace
sid <- H5Screate_simple(c(10,5,3))

## Select elements that lie in in the rows 1-3, columns 2-4, 
## and the entire 3rd dimension
H5Sselect_index(sid, list(1:3, 2:4, NULL))

## We can check the number of selected points. 
## This should be 27 (3 * 3 * 3)
H5Sget_select_npoints(sid)

## always close dataspaces after usage to free resources
H5Sclose(sid)

Description

Set the selection region of a dataspace to include no elements

Usage

H5Sselect_none(h5space)

Arguments

Description

Check that a selection is valid

Usage

H5Sselect_valid(h5space)

Arguments

Description

Set the size of a dataspace

Usage

H5Sset_extent_simple(h5space, dims, maxdims)

Arguments

Description

The value for H5S_UNLIMITED can be provided to the maxdims argument of H5Screate_simple to indicate that the maximum size of the corresponding dimension is unlimited.

Usage

H5Sunlimited()

See Also

H5Screate_simple

Description

Retrieve or set the character set to be used in a string datatype.

Usage

H5Tset_cset(dtype_id, cset = "ASCII")

H5Tget_cset(dtype_id)

Arguments

Description

Create or modify an HDF5 enum datatype

Usage

H5Tenum_create(dtype_id = "H5T_NATIVE_INT")

H5Tenum_insert(dtype_id, name, value)

Arguments

Value

• H5Tinsert_enum() returns an character representing the H5 identifier of the new datatype.

• H5Tset_precision() is called for its side-effect of modifying the existing datatype. It will invisibly return TRUE if this is successful FALSE if not.

Examples

tid <- H5Tenum_create(dtype_id = "H5T_NATIVE_UCHAR")
H5Tenum_insert(tid, name = "TRUE", value = 1L)
H5Tenum_insert(tid, name = "FALSE", value = 0L)

Description

Get details of HDF5 data types

Usage

H5Tget_class(dtype_id)

H5Tget_nmembers(dtype_id)

Arguments

Value

• H5Tget_class() returns an character vector of length 1 giving the class of the data type.

• H5Tget_nmembers() returns the number of members in the given datatype. Will fail with an error if the supplied datatype is not of type H5T_COMPUND or H5T_ENUM.

Examples

## create an enum datatype with two entries
tid <- H5Tenum_create(dtype_id = "H5T_NATIVE_UCHAR")
H5Tenum_insert(tid, name = "TRUE", value = 1L)
H5Tenum_insert(tid, name = "FALSE", value = 0L)

H5Tget_class(tid)
H5Tget_nmembers(tid)

Description

Retrieve or set the precision of an HDF5 datatype

Usage

H5Tset_precision(dtype_id, precision)

H5Tget_precision(dtype_id)

Arguments

Value

• H5Tget_precision() returns an integer giving the number of significant bits used by the given datatype.

• H5Tset_precision() is call for its side-effect of modifying the precision of a datatype. It will invisibly return TRUE if this is successful and will stop with an error if the operation fails.