nuget
Hierarchical Data Stream Pack, A hdf5 liked file format
StreamPack
01 Syntax
Microsoft.VisualBasic.DataStorage.HDSPack.FileSystem.StreamPack
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| .ctor | 2 | |
| Clear | 1 | Clear the file tree |
| SetAttribute | 2 | set global attributes |
| ToString | 1 | get debug view of current data pack object |
| ParseTree | 1 | parse the magic header/global metadata attribute/filesystem tree |
| GetObject | 1 | Get target object and its corresponding attributes data which is specified by the given fileName |
| OpenBlock | 2 | Open an existed block file |
| FileExists | 1 | Check of the given file data stream is existsed inside current data pack file |
| OpenFile | 1 | a more advanced wrapper for StreamPack.OpenBlock()) function |
| Allocate | 1 | try allocate a free block inside the file, not append to it |
| OpenReadOnly | 1 | Open a stream pack data file for read data |
| flushStreamPack | 1 | just write stream header data |
| Dispose | 1 | 已经在dispose函数之中处理好所有文件数据保存，以及文件释放的工作，直接dispose即可 |
| Close | 1 | Save the file tree and close the file stream |
| WriteText | 1 | |
| Flush | 1 | Commit the metadata header stream into filesystem |
| GetFiles | 1 | Get all files inside the given folder path |
| Delete | 1 | |
| GetGlobalAttribute | 1 | |
| TestMagic | 1 | |
| OpenFolder | 1 |
03 Properties
| Name | Overloads | Summary |
|---|---|---|
| filepath | 1 | gets the file path of the underlying stream data of current stream pack object |
| files | 1 | get all data files inside this hds data pack, not includes directory. |
| HeaderSize | 1 | the actual used file header size |
| superBlock | 1 | |
| globalAttributes | 1 | |
| is_readonly | 1 | |
| isDiskFile | 1 |
04 Fields
| Name | Overloads | Summary |
|---|---|---|
| buffer | 1 | usually be the underlying local file stream for read/write pack data |
| _registriedTypes | 1 | the type list of the values in the StreamPack.globalAttributes data, a messagepack schema should be defined for these types. |
| _meta_size | 1 | the pre-allocated metadata header size, includes of the global metadata and filesystem tree |
| Magic | 1 |
05 Members
#ctor(
String, Int32, Int64, Boolean, Boolean)open or create new stream pack data object
Parameters
| Name | Type | Description |
|---|---|---|
filepath | String | - |
init_size | Int32 | - |
meta_size | Int64 | default 1MB for save the header data |
#ctor(
Stream, Int32, Int64, Boolean)Parameters
| Name | Type | Description |
|---|---|---|
buffer | Stream | - |
init_size | Int32 | the initialize block file of the memorystream for the block file when do create new file. |
meta_size | Int64 | the size in bytes of the tree header data, used for create new file only |
Clear(
Int64)Clear the file tree
Parameters
| Name | Type | Description |
|---|---|---|
meta_size | Int64 | the size in bytes of the tree header data |
SetAttribute(
NamedValue(Of Object)())set global attributes
Parameters
| Name | Type | Description |
|---|---|---|
attrs | NamedValue(Of Object)() | - |
SetAttribute(
String, Dictionary(Of String, Object))Parameters
| Name | Type | Description |
|---|---|---|
fileName | String | the dir object its file name must be ends with the symbol '\' or '/' |
attrs | Dictionary(Of String, Object) | all attribute value will be serialized via messagepack |
ToString
get debug view of current data pack object
ParseTree
parse the magic header/global metadata attribute/filesystem tree
GetObject(
String)Get target object and its corresponding attributes data which is specified by the given fileName
Parameters
| Name | Type | Description |
|---|---|---|
fileName | String | the dir object its file name must be ends with the symbol '\' or '/' |
Returns
returns nothing if object is not found!
OpenBlock(StreamBlock,
Boolean)Open an existed block file
Parameters
| Name | Type | Description |
|---|---|---|
block | StreamBlock | - |
Returns
this function will returns a MemoryStream in-memory data when loadMemory parameter value is set to TRUE.
OpenBlock(
String, Int32)open a data block for read and write
if the target file block is missing from the tree, then this function will append a new file block based on the StreamPack.is_readonly flag is set to false or not, otherwise a substream object will be returned for read data
Parameters
| Name | Type | Description |
|---|---|---|
fileName | String | the dir object its file name must be ends with the symbol '\' or '/' |
buffer_size | Int32 | options for write data only |
Returns
this function returns two type of the stream:
SubStreamfor readonly, this kind of stream should not call the dispose method, or the based file stream will be closed too!- StreamBuffer for writeonly, this kind of stream must be use the dispose method for commit data to based file stream
based on the target file object is existsed or not
FileExists(
String, Boolean)Check of the given file data stream is existsed inside current data pack file
Parameters
| Name | Type | Description |
|---|---|---|
path | String | - |
ZERO_Nonexists | Boolean | this parameter is not working at here |
OpenFile(
String, FileMode, FileAccess)a more advanced wrapper for StreamPack.OpenBlock() function
Remarks
the parameter access only supports data mode:
FileAccess.Read;FileAccess.Write;
Do not dispose the readonly stream; for a write stream, dispose method should be call for commit the data into filesystem.
this function create data buffer without pre-allocation for write, and append to the file last location
Parameters
| Name | Type | Description |
|---|---|---|
path | String | - |
mode | FileMode | this parameter is no use in streampack |
access | FileAccess | - |
Allocate(
Int32)try allocate a free block inside the file, not append to it
Remarks
this function may be too slow if too much file to process
Parameters
| Name | Type | Description |
|---|---|---|
buffer_size | Int32 | - |
OpenReadOnly(
String, Boolean)Open a stream pack data file for read data
Parameters
| Name | Type | Description |
|---|---|---|
filepath | String | - |
strict_missing | Boolean | this parameter controls the behaviours about the missing filepath, if this parameter is set to value true(default behaviours), function will throw file not found exception, otherwise will returns an empty stream object. |
Returns
this function may returns an empty stream data if the given filepath is missing on the filesystem andalso not set strict_missing to true.
flushStreamPack
just write stream header data
Remarks
check of the header size is larger than the pre-allocated metadata size, and populate warning message if it is.
Dispose
已经在dispose函数之中处理好所有文件数据保存，以及文件释放的工作，直接dispose即可
Close
Save the file tree and close the file stream
WriteText(
String, String)Remarks
this method is a safe method, it alwasy delete the target file stream at first and then do text write
Parameters
| Name | Type | Description |
|---|---|---|
text | String | - |
path | String | - |
Flush
Commit the metadata header stream into filesystem
GetFiles(
String)Get all files inside the given folder path
Parameters
| Name | Type | Description |
|---|---|---|
dir | String | - |
Returns
this function returns an empty collection of the file path string if the given dir is not exists in the archive file tree or its value is nothing
filepath
gets the file path of the underlying stream data of current stream pack object
Returns
this property will returns the file path string if the underlying stream data is StreamPack.isDiskFile, otherwise nothing will be returns.
files
get all data files inside this hds data pack, not includes directory.
HeaderSize
the actual used file header size
buffer
usually be the underlying local file stream for read/write pack data
_registriedTypes
the type list of the values in the StreamPack.globalAttributes data, a messagepack schema should be defined for these types.
_meta_size
the pre-allocated metadata header size, includes of the global metadata and filesystem tree
Magic
superBlock
globalAttributes
is_readonly
isDiskFile
Delete(
String)GetGlobalAttribute(
String)TestMagic(
Byte())OpenFolder(
String)