Organization of .umt files

A .umt file is umIT's container for results that are described by named dimensions. Many results do not fit the image dimensions of a .dat file (Y, X, T, E and F): an ROI trace has an ROI axis, a correlation matrix has two ROI axes, and a result combined across recordings has no single recording to belong to. A UMT file stores such results together with their own dimension names, labels, event information, and processing information. It can also hold image data, and DataViewer can open compatible image-kind UMT entries.

Which of the two formats an analysis function produces is decided by the function's author, not by the person running it. Authors: see Choosing the output format.

Where to find what

Everything a UMT file contains is a MATLAB structure with the following layout. The table shows where each kind of information lives and how to reach it after loading the file with umt = loadData('file.umt').

Information	Location	Shared by all entries?
Measured values	umt.data.<entryName>.value	No, one per entry
Meaning of each dimension of the values	umt.data.<entryName>.dimNames	No, one per entry
Processing information of one entry (for example its frame rate)	umt.data.<entryName>.meta (optional)	No, one per entry
Names for the positions along a dimension (for example ROI names or measure names)	umt.labels.<dimension> (optional)	Yes
Events that the E dimension refers to	umt.eventInfo (only with an E dimension)	Yes
Kind of container and format version	umt.kind, umt.version	Yes

Top-level structure (version 1)

Field	Required?	Meaning
version	Yes	UMT format version used during validation.
kind	Yes	image or roi; determines which combinations of dimensions are accepted.
data	Yes	Structure containing one or more named measurement entries.
labels	No	Shared display/reference labels.
eventInfo	For event-axis data	Shared condition names, identifiers, repetitions, and optional baseline duration.

Entries are measurements

Think of each data.<entryName> as one measure derived from a dataset. For example, a single UMT file could contain an ROI-by-time fluorescence trace, an ROI-by-measure response summary, and an ROI-by-ROI correlation matrix. These entries can have distinct dimensions because each entry declares its own:

Entry field	Meaning
value	The numeric or logical measurement.
dimNames	The ordered meaning of every dimension, using names such as Y, X, T, E, F, ROI, Pixel, or Measure.
meta	Optional processing information specific to that measurement. The only field with a defined meaning is FrameRateHz, the frame rate of that entry's own data, where applicable.

The number and order of dimNames must match value. Every entry must also match a supported pattern for its UMT kind. The accepted dimension names are Y, X, T, E, F, ROI, Pixel, and Measure.

Allowed dimension patterns for kind image

Pattern	Meaning
{Y,X}	One image.
{Y,X,T}	Image time series.
{Y,X,E}	One image per event (or per event condition after averaging).
{Y,X,F}	One image per F entry, for example amplitude and phase maps.
{Y,X,T,E}	Image time series split by event.
{Y,X,ROI}	One image per ROI.

Allowed dimension patterns for kind roi

An entry of an roi UMT must use one of the following dimNames patterns:

Pattern	Meaning
{ROI}	One value per ROI.
{ROI,T}	One time series per ROI.
{ROI,E}	One value per ROI and per event.
{ROI,F}	One value per ROI and per F entry, as produced by getDataFromROI from Y-X-F maps.
{ROI,Measure}	One value per ROI and per measure.
{ROI,Measure,E}	One value per ROI, measure and event.
{ROI,T,E}	One time series per ROI and per event.
{ROI,Pixel}	One value per pixel of each ROI.
{ROI,Pixel,T}	One time series per pixel of each ROI.
{ROI,Pixel,E}	One value per pixel of each ROI and per event.
{ROI,Pixel,T,E}	One time series per pixel of each ROI and per event.
{ROI,ROI}	ROI-by-ROI matrix, such as a correlation matrix.

There is no {ROI,Pixel,F} pattern; getDataFromROI therefore does not accept SpatialAggFcn='none' for inputs with an F dimension.

Labels

labels holds one list of text labels per dimension, named after the dimension (for example labels.ROI or labels.Measure). A list must have exactly as many elements as the dimension has positions in every entry that uses it, and a labeled dimension must be used by at least one entry. Labels are for display and reference. Do not rely on them for calculations: use the position along the dimension.

Event information

eventInfo exists only when at least one entry has an E dimension, and all entries with an E dimension must have the same E length. Each field has one element per position along E:

Field	Meaning
eventID	Condition ID for each E-axis element.
repetitionIndex	Repetition number, or zero for a result already aggregated across repetitions.
eventName	Condition name corresponding to each E-axis element.
eventAxisMode	instances (each E element is one occurrence of an event) or aggregated_repetitions (each E element is one condition, already combined over its repetitions).
baselinePeriod	Optional baseline duration in seconds. The baseline is the part of each event window that precedes the stimulus. Functions that convert times to frames, such as calculateResponseFeatures, treat the first frame after the baseline as time zero.
selected	Optional. Logical, one value per E element. It is false for event instances ignored in the Events Manager. For aggregated elements it is true when the slice holds data.
durationSec	Optional. Duration of each event in seconds, from its ON to its OFF transition, or NaN.
nInstances	Optional, only for aggregated_repetitions. Number of selected event instances combined into each element.

See Events for how events are created and selected.

Per-entry processing information

eventInfo and labels describe the whole file, so every entry that uses E or a labeled dimension shares them. meta belongs to one entry only. meta.FrameRateHz is the frame rate of the time axis of that entry's own data. It is a positive number, it is present only where it applies (an image or a map without time has none), and it is never taken from AcqInfos.mat. Functions that convert seconds to frames, such as calculateResponseFeatures, stop with an error when it is missing. An analysis function that keeps the time axis (for example getDataFromROI) carries meta and eventInfo over to its output unchanged.

How a .umt file is stored

A .umt file is a MATLAB MAT-file (version 7.3) with a custom extension that holds the fields of the structure above. umIT always reads it completely: loadData and the DataViewer load the whole file into the computer's memory. This is no problem for the small tables, matrices, and maps that UMT files usually hold. A very large image time series saved as .umt can be slow to open or too large to load, which is why such data are stored as .dat files, which can be read in parts.

Reading and using .umt files in MATLAB

The three examples below create a small practice recording, run three Stats Functions on it, and read their outputs. They use only functions of the toolbox (the repository folders must be on the MATLAB path) and a temporary folder. When working with your own data, skip the practice-data block and point datFile and saveFolder to your own Save Folder, which must contain the ROI file (myROI.roi by default).

Practice data

% A 20-by-30-pixel recording of 100 frames at 10 Hz, saved as a .dat file.
saveFolder = fullfile(tempdir, 'umt_practice');
mkdir(saveFolder);
rng(1);
frameRateHz = 10;
img = randn(20, 30, 100, 'single');                 % Y x X x T
img(3:8, 3:8, 41:60) = img(3:8, 3:8, 41:60) + 3;    % a response in the area of ROI1
datFile = fullfile(saveFolder, 'practice.dat');
saveData(datFile, img, 'DimNames', {'Y','X','T'}, 'FrameRateHz', frameRateHz);

% Two rectangular ROIs saved in an ROI file (normally drawn in DataViewer).
roi1 = makeRectROI('ROI1', [20 30], 3:8, 3:8);
roi2 = makeRectROI('ROI2', [20 30], 12:18, 20:28);
saveROIFile(fullfile(saveFolder, 'myROI.roi'), createROIFile([20 30], 'ROIs', [roi1, roi2]));

function roi = makeRectROI(name, imageSizeYX, rows, cols)
% Minimal ROI with the fields required by the .roi format.
mask = false(imageSizeYX);
mask(rows, cols) = true;
x = [min(cols) max(cols) max(cols) min(cols)] + [-0.5 0.5 0.5 -0.5];
y = [min(rows) min(rows) max(rows) max(rows)] + [-0.5 -0.5 0.5 0.5];
vertices = [x(:) y(:)];
stats = struct('computedOn', datetime('now'), 'NPixels', nnz(mask), 'areaPx2', nnz(mask), ...
    'areaMM2', [], 'centroidXY_px', [], 'centroidXY_mm', [], ...
    'distanceFromOrigin_px', [], 'distanceFromOrigin_mm', [], 'spatialMean', [], ...
    'spatialStd', [], 'spatialMedian', [], 'spatialMin', [], 'spatialMax', []);
roi = struct('name', name, 'type', 'polygon', 'DOC', datetime('now'), ...
    'modifiedOn', datetime('now'), 'color', [1 0 0], 'notes', '', ...
    'geometry', struct('polyshape', polyshape(vertices(:,1), vertices(:,2)), ...
        'verticesXY_px', vertices, 'ROIType', 'polygon', ...
        'ROIParameters', struct('ROIType', 'polygon', 'Position', vertices, 'Vertices', vertices)), ...
    'mask', mask, 'stats', stats);
end

In a MATLAB script, local functions such as makeRectROI must be at the end of the file.

Example 1: ROI time courses from getDataFromROI

getDataFromROI returns a UMT of kind roi. With a .dat input, its single entry is named main.

roiUMT = getDataFromROI(datFile, saveFolder);        % reads myROI.roi from saveFolder
saveData(fullfile(saveFolder, 'ROI_data.umt'), roiUMT);   % in a pipeline, PipelineManager saves it

umt = loadData(fullfile(saveFolder, 'ROI_data.umt'));
umt.kind                        % 'roi'
fieldnames(umt.data)'           % {'main'}
entry = umt.data.main;
entry.dimNames                  % {'ROI','T'}
size(entry.value)               % [2 100]: 2 ROIs by 100 frames
umt.labels.ROI                  % {'ROI1','ROI2'}
entry.meta.FrameRateHz          % 10

% Use it: plot every ROI trace against time in seconds.
t = (0:size(entry.value, 2) - 1) / entry.meta.FrameRateHz;
plot(t, entry.value.');
legend(umt.labels.ROI); xlabel('Time (s)');

The dimension order in entry.dimNames is the order of the dimensions of entry.value, so ROIs run down the rows and frames across the columns. No eventInfo exists because the data have no E dimension.

Example 2: response features per event from calculateResponseFeatures

calculateResponseFeatures needs a ROI UMT with dimensions {'ROI','T','E'}, event information, and a frame rate. In a real analysis, split_data_by_event produces the event-split data; here an equivalent image UMT is built by hand, with its event information, to keep the example self-contained.

% Image time series split into 6 events (3 repetitions of 2 conditions).
nT = 100; nE = 6;
trials = randn(20, 30, nT, nE, 'single');
trials(3:8, 3:8, 21:50, :) = trials(3:8, 3:8, 21:50, :) + 3;
names = {'Left'; 'Left'; 'Left'; 'Right'; 'Right'; 'Right'};
eventInfo = struct( ...
    'eventID', [1; 1; 1; 2; 2; 2], ...
    'repetitionIndex', [1; 2; 3; 1; 2; 3], ...
    'eventName', {names}, ...
    'eventAxisMode', 'instances', ...
    'baselinePeriod', 2);                                  % seconds
trialUMT = genUMTStruct(trials, 'kind', 'image', 'entryName', 'main', ...
    'dimNames', {'Y','X','T','E'}, 'meta', struct('FrameRateHz', frameRateHz), ...
    'eventInfo', eventInfo);

% Extract the ROI traces, save them, and calculate the features.
roiTrials = getDataFromROI(trialUMT, saveFolder);
roiFile = fullfile(saveFolder, 'ROI_data_events.umt');
saveData(roiFile, roiTrials);
feat = calculateResponseFeatures(roiFile, 'STD_threshold', 2);

entry = feat.data.main;
entry.dimNames                  % {'ROI','Measure','E'}
size(entry.value)               % [2 6 6]: ROIs by measures by events
feat.labels.Measure             % PeakAmplitude, PeakLatency, TimeWindowAverageAmplitude,
                                % AUCamplitude, OnsetAmplitude, OnsetLatency
feat.eventInfo.eventAxisMode    % 'instances': one E element per event

% Use it: mean peak amplitude of ROI1 for each condition.
iPeak = find(strcmp(feat.labels.Measure, 'PeakAmplitude'));
peak = reshape(entry.value(:, iPeak, :), size(entry.value, 1), []);   % ROI x E
conditions = unique(feat.eventInfo.eventName, 'stable');
for k = 1:numel(conditions)
    isCond = strcmp(feat.eventInfo.eventName, conditions{k});
    fprintf('%s: ROI1 mean peak = %.2f\n', conditions{k}, mean(peak(1, isCond)));
end

The output keeps the E dimension of the input, so feat.eventInfo still describes every position along E. The event-name list is how you group the events by condition; the position along E is what links a value to its event.

Example 3: correlation matrix and seed maps from genCorrelationMatrix

genCorrelationMatrix returns a ROI UMT with an ROI-by-ROI matrix. With b_genSPCMaps it also saves an image UMT with one Y-X correlation map per ROI, using each ROI as the seed.

[corrUMT, spcFile] = genCorrelationMatrix(datFile, saveFolder, 'b_genSPCMaps', true);

cm = corrUMT.data.CorrMatrix;
cm.dimNames                     % {'ROI','ROI'}
size(cm.value)                  % [2 2]
corrUMT.labels.ROI              % {'ROI1','ROI2'}: names of rows and columns

% The seed-pixel maps: an image UMT with one Y-X entry per ROI, named after the ROI.
spc = loadData(fullfile(saveFolder, spcFile));
spc.kind                        % 'image'
fieldnames(spc.data)'           % {'ROI1','ROI2'}
spc.data.ROI1.dimNames          % {'Y','X'}
imagesc(spc.data.ROI1.value);   % map of correlation with the seed of ROI1

Create, append, and validate

Normally an analysis function constructs a UMT result and PipelineManager passes it to the next step or saves it when required. For advanced MATLAB use, create and append entries with genUMTStruct and save them with saveData, which validates the structure before writing it:

% Create an image time-series measurement (frame rate in the entry's meta).
umt = genUMTStruct(imageData, ...
    'kind', 'image', ...
    'entryName', 'correctedFluorescence', ...
    'dimNames', {'Y','X','T'}, ...
    'meta', struct('FrameRateHz', 10));

% Append a Y-X amplitude measurement to the same container.
umt = genUMTStruct(umt, ...
    'value', amplitudeMap, ...
    'entryName', 'peakAmplitude', ...
    'dimNames', {'Y','X'});

validateUMTStruct(umt, 'requireEventInfo', false);
saveData(fullfile(saveFolder, 'processedResults.umt'), umt);

Use appendUMTEventInfo when one or more entries declare an E dimension. Validation requires all E-based entries to use the same E length and the event vectors to match it.

Project identity stays outside the UMT file
A UMT file can carry scientific labels, dimensions, events, and processing information. Project, subject, and session organization remains in the central project information and the Save Folder link; folder-wide masks and Raw Folder paths remain in DataParams.mat.

Previous data and manual editing

Processed data created with the previous release, especially results saved as general .mat files, may not be compatible with the UMT structure. Load and convert those results deliberately; changing the extension does not convert the data.

Validate every UMT loaded or changed in MATLAB. Direct field editing is unsupported because a small change can make entry dimensions, labels, or event information disagree.