PLW file date format

Forum for discussing PicoLog
Jim Howes

PLW file date format

Post by Jim Howes »

I am trying to grok the PLW file format as described on page 101 of the plw.pdf manual.

Two problems. The first I think I have worked out. The line
UNS16 parameters[50];

should, I think, read
UNS16 parameters[250];

or perhaps I am missing something else there? Either way nothing makes sense unless I sling an extra 200 bytes in there. There is also a highly mysterious gap between UNS32 start_time; and INT32 minimum_time; is something else missing?

Secondly, I am trying to work out how the start_date is encoded, without a lot of luck. The start time appears to be seconds past midnight, but I have not yet sussed the date format. Ideally, I need it as a C time_t

Regards,
Jim

User avatar
markspencer
Site Admin
Site Admin
Posts: 598
Joined: Wed May 07, 2003 9:45 am

Post by markspencer »

Hi,

Unfortunately, I am unable to provide the information you are requiring. The .plw file has changed and is likely to keep changing which means that the details in the help manual are out of date.

We are intending to rectify this in the near future, but I do not have a timescale for this.

Best regards,
Regards,

Mark Spencer

dgibert

PLW files format

Post by dgibert »

I also URGENTLY need to read binary PLW files on my Linux Computer. I agree that the users' guide concerning the PLW format is wrong.

Please, give us the right PLW format. A C or a MATLAB procedure for PLW reading would be welcome.

Thanks,

Dominique Gibert

User avatar
markspencer
Site Admin
Site Admin
Posts: 598
Joined: Wed May 07, 2003 9:45 am

Post by markspencer »

Hi,

Unfortunately, I am unable to provide this information at present.

Best regards
Regards,

Mark Spencer

Jim Howes

Re: PLW files format

Post by Jim Howes »

dgibert wrote:I also URGENTLY need to read binary PLW files on my Linux Computer. I agree that the users' guide concerning the PLW format is wrong.
As it happens, I've finished hacking around with my code and have something that works, atleast with the version of PLW files that my implementation creates.

My declaration reads:

Code: Select all

#pragma pack(1)  // Stupid alignment follows
typedef struct PICO_header_struct
{
	unsigned short	header_bytes;
	char		signature[40];
	unsigned long	version;
	unsigned long	no_of_parameters;
	unsigned short	parameters[250];	// !! The value in the PICOlog documentation is entirely bogus!
	unsigned long	sample_no;
	unsigned long	no_of_samples;
	unsigned long	max_samples;
	unsigned long	interval;
	unsigned short	interval_units;
	unsigned long	trigger_sample;
	unsigned short	triggered;
	unsigned long	first_sample;
	unsigned long	sample_bytes;
	unsigned long	settings_bytes;
	unsigned long	start_date;
	unsigned long	start_time;
	long		minimum_time;
	long		maximum_time;
	char		notes[200];
	long		current_time;
	unsigned char	spare[78];
} PICO_header_t;
#pragma pack()
Note that this is for MS Visual C, For GNU C, use #pragma align 1 if you have structure alignment problems, which I suspect you may have.

I process the date and time into a C time_t as follows

Code: Select all

	time_t	tBaseTime;
	struct tm *tmDate;
	PICO_header_t	PH;

   ...stuff to open file and fread((void *)&PH,sizeof(PH),1,inputfile)...

	tBaseTime = 86400 * (PH.start_date - 719163) + PH.start_time; 
	tmDate = gmtime(&tBaseTime);
Note I've used gmtime(). However, I think this might be wrong. Windows applications are notoriously bad at handling time zones properly in my opinion; comes from a history of assuming that your user is sat infront of the PC all the time; a luxury us unix hackers have never had to put up with. Quite what happens when your log file includes times during the changeover from/to DST, I don't know. localtime() is another function you could try.

Before reading data, I have to

Code: Select all

	fseek(input, PH.header_bytes, SEEK_SET);
because sizeof(PH) appears to bear absolutely no relation to the actual layout of the file because the documentation is entirely bogus in that regard. The suspicious gap between start_time and minimum_time in the documentation suggest that the whole story isn't being told.

Anyway, make of the above what you will. It comes with no warranty, and if it breaks you get to keep both pieces.

Regards,
Jim

dgibert

Post by dgibert »

Thank you very much Jim,

indeed the Pico indications are quite false ! I'll try your code on my files.

Beyond this, I am surprised by the Moderator response. How is it possible to ignore the PLW format ? Does my picolog software version write PLW files without knowing the exact file format. I am EFFRAID !!!

Regards and thanks again,

Dominique

P. F. R.

plw file format

Post by P. F. R. »

hi all,

I've worked out the plw file format. here is it:
unsigned integer, 16 bit: uint16
unsigned integer, 32 bit: uint32

uint16 header size
char[40] signature
uint32 version
uint32 no. of parameters
uint16[250] parameters
uint32 sample no.
uint32 sample no. (twice)
uint32 max sample
uint32 interval
uint16 interval units
uint32 trigger sample
uint16 triggered
uint32 first sample
uint32 sample byte
uint32 setting byte
uint32 start date
uint32 start time
uint32 min time
uint32 max time
char[1000] notes
uint32 current time
uint8[78] spare

that's it! it gives a total header size of 1684 bytes, which seems to be alright.note that the informations given in the manual are wrong on the size of the array "parameters" and "notes".
hope this will be useful...
cheers!
Pierre-François

P.F.R.

start_time

Post by P.F.R. »

Hi all,
I've also found that the start_time parameter was simply...the number of seconds elapsed in one day (starting at midnight).
sounds simple, isn't it? :D
Bless bless

TK

PLW DATA FILE

Post by TK »

hi
could some one help me please. I need to know exactly what PLW files are, how to read and use them. I have not been able to understand the user manul of PICO LOG. please e-mail me at teekay_tk@hotmail.com
i shall be grateful
regards

taimur

User avatar
markspencer
Site Admin
Site Admin
Posts: 598
Joined: Wed May 07, 2003 9:45 am

Post by markspencer »

Hi,

I am sorry to hear that you are experiencing problems with understanding the layout for the PLW files. Unfortunately, the help manual is not totaly up to date on the structure used, but if you look at P.F.R post of 16 Apr 04, this gives the correct format. Now that you know this you can use this structure to read the byes out of the file.

Best regards,
Regards,

Mark Spencer

leepatrk
Newbie
Posts: 1
Joined: Sun Mar 19, 2006 12:47 am

Post by leepatrk »

Any updates on this? I tried the format above by P.F.R and it doesn't quite align. I'm using the latest version of Picolog: 5.13.7

Is the order of data still the same?

THanks,
Lee

Sarah

Post by Sarah »

Hi Lee

No I believe the file format has changed since that post was written. Does the help manual provide you with the information that you require?

Best Regards

LarsW
User
User
Posts: 6
Joined: Thu Mar 23, 2006 9:04 am
Location: Lund University

Post by LarsW »

Some time ago I wrote a MATLAB-program to read PLW-files directly into MATLAB. I remember that the manual did not give the whole truth, but also that (if I remember correctly) different versions (I found three) were slightly different. So if you want a program that works for all PLW-files you need to check for "version" have slightly different code for different version.

LarsW
User
User
Posts: 6
Joined: Thu Mar 23, 2006 9:04 am
Location: Lund University

Post by LarsW »

Here is my program PLW2ML.m that converts PLW-files to MATLAB variables. On lines 59-69 you can see the differences I noted between the different versions 1, 2 and 3. Note that the param.PLS, which contains information about the run (for example the names given to the channels) had an initial part that I could not decipher. I wrote this program ago year and a half ago and have used it to evaluate files in an automated way. Look at the examples in the initial comments (starting with %). I especially liked the the way I evaluated a large number of PLW-files, each with measurements on eight randomly chosen samples, and then found which sample that had been in each channel in each run by looking that up in param.PLS (In each run I named the channels HDMXX, with XX being the number of the sample).

Yesterday I tested the program on some files of version 2 and 3 I found in my computer. For many files it worked well and was quick, but for some version 2 files the reading of the data (the loop in lines 114-117) took several minutes even for not too large data files. It could probably be improved by using some more efficient way of reading the data matrix into MATLAB. So I guess the MATLAB code is not optimal AND it may even not be correct for all types of PLW-files...(is there a version 4?).

Here is the code (the Picotech Forum would not allow me to attach an m-file, so I had to paste it into the message)

function [t,U,param]=PLW2ML(fullfilename,info)
% PLW2ML converts PLW-files (PicoTech Picologger) to MATLAB variables
% [t,U,param]=PLW2ML(filename,info)
% filename is the full filename (with path and extension .plw)
% info is an optional parameter that if set to 1 displays
% information about the file being read
% t is the time vector
% U is the output (voltage, temperature etc. vector/matrix)
% param is a structure containing other information,
% for example param.PLS is a settings file (first part is strange)
%
% A typical use of the plw2ml function is as follows:
%
% [t,U]=plw2ml('c:\measure2\rotfungi\sl0921A.plw');
% plot(t,U(:,1))
%
% A more advanced way is to use it on all files of a certain sort,
% for example to plot the data in all files starting with 'sl' in one directory:
%
% presdir=cd; %present directory
% dirvec=dir; %Find all filenames in directory
% for k=1:length(dirvec) %go through all files
% filename=dirvec(k).name; %put filename in filename-string
% if length(filename)>5 % do not evaluate '-' '--' and other strange things in dirvec
% if strcmp(filename(1:2),'SL') %only if it is a SLXXXX- file
% fullfilename=[presdir,'\',filename]; %full filename, incl. path
% [t,U]=PLW2ML(fullfilename);
% plot(t,U);
% end
% end
% end
%
% An even more automated evaluation method can be made by looking up,
% e.g., sample identifiers in the param.PLS-variable (if they have
% been entered into PLW before the start of the measurements). In the
% following example samples (channels) are named HDMXX, where XX is a
% sample number that the program finds and uses to sort the measurements
%
% k=k+1;
% indstart=findstr('MultiConverter',param.PLS); %---Find line before channel identifier
% indname=findstr('HDM',param.PLS(indstart:end)); %---In this example sample numbers follow HDM-string
% %find all places in PLS-file starting with 'Name=HDM' (that are followed by the sample number)
% for p=1:length(indname) %for all measurements found in the file
% sample_no(k)=str2num(param.PLS(indstart+indname(p)+2:indstart+indname(p)+3)); %extract sample number
% end
%
% Lars Wadsö, Building Materials, Lund University, Sweden 28 Oct 2004

if nargout<2|nargout>3;error('PLW2ML needs two or three output arguments [t,U] or [t,U,param]');end
if nargin==0;[fn,pn]=uigetfile('*.*','Open a measurement file');fullfilename=[pn,fn];info=0;end
if nargin==1;info=0;end
if fullfilename(end-3:end)~='.plw' & fullfilename(end-3:end)~='.PLW';error('PLW2ML can only open files with extension .plw or .PLW');end
fid=fopen(fullfilename); %open file for reading
if fid==-1;error('File could not be opened');end
param.header_bytes=fread(fid,1,'ushort'); %read first line of PLW-HEADER (the end of the file contains a partial explanation of how a PLW file is built up)
param.signature=char(fread(fid,40,'uchar'))'; %etc
param.version=fread(fid,1,'uint32');
if info;disp(['PLW file version ',int2str(param.version)]);end
switch param.version
case 1
antal_parametrar=50;
antal_notes=200;
case 2
antal_parametrar=50; %100;
antal_notes=200;
case 3
antal_parametrar=250;
antal_notes=1000;
end
param.no_of_parameters=fread(fid,1,'uint32');
if info;disp(['Number of parameters ',int2str(param.no_of_parameters)]);end
param.parameters=fread(fid,antal_parametrar,'uint16'); %says 50 in PLW manual
if info;disp(['Parameters: ',int2str(param.parameters(1:param.no_of_parameters)')]);end
% param.sample_no=0;
% while param.sample_no==0
param.sample_no=fread(fid,1,'uint32'); %=following
% end
if info;disp(['Number of samples: ',int2str(param.sample_no)]);end
param.no_of_samples=fread(fid,1,'uint32'); %=previous %number of samples
if info;disp(['No OF SAMPLES: ',int2str(param.no_of_samples)]);end
param.max_samples=fread(fid,1,'uint32');
if info;disp(['MAX Number of samples: ',int2str(param.max_samples)]);end
param.interval=fread(fid,1,'uint32'); %measurement interval
if info;disp(['Interval: ',int2str(param.interval)]);end
param.interval_units=fread(fid,1,'uint16'); %interval units
if info;disp(['Interval_units: ',int2str(param.interval_units)]);end
switch param.interval_units
case 0;units=' fs';
case 1;units=' ps';
case 2;units=' ns';
case 3;units=' us';
case 4;units=' ms';
case 5;units=' s';
case 6;units=' min';
case 6;units=' h';
otherwise;units=' with unknown unit';
end
if info;disp(['Sampling interval: ',num2str(param.interval),units]);end
param.trigger_sample=fread(fid,1,'uint32');
param.triggered=fread(fid,1,'uint16');
param.first_sample=fread(fid,1,'uint32');
param.sample_bytes=fread(fid,1,'uint32');
param.settings_bytes=fread(fid,1,'uint32');
param.start_date=fread(fid,1,'uint32'); %start date (days since start of year 0)
if info;disp(['Start date (days since 1 jan year 0) ',int2str(param.start_date)]);end
param.start_time=fread(fid,1,'uint32'); %start time (seconds since start of day)
if info;disp(['Start time (secondas since beginning of day) ',int2str(param.start_time)]);end
param.minimum_time=fread(fid,1,'int32');
param.maximum_time=fread(fid,1,'int32');
param.notes=fread(fid,antal_notes,'uchar')';
param.current_time=fread(fid,1,'int32');
param.spare=fread(fid,78,'uint8');
%read DATA
for p=1:param.no_of_samples %here the samples are read
t(p)=fread(fid,1,'uint')'; %time
U(p,:)=fread(fid,param.no_of_parameters,'float'); %no_of_parameters of data
end
%read PLS-file appended at end of PLW-file (see the end of this file for an explanation)
param.PLS=char(fread(fid,inf,'uchar'))'; %here the PLS-file is read (the first part contains strange text)
if info;disp('The param.PLS file with infomation about the run can be retreived from a third output argument');end
fclose(fid); %close file

Guest

Post by Guest »

Thanks for the code Lars, i made some improvements for it;

- Support for v4
- Increased speed !!

Code: Select all

function [t,U,param]=PLW2ML(fullfilename,info) 
% PLW2ML converts PLW-files (PicoTech Picologger) to MATLAB variables 
% [t,U,param]=PLW2ML(filename,info) 
% filename is the full filename (with path and extension .plw) 
% info is an optional parameter that if set to 1 displays 
% information about the file being read 
% t is the time vector 
% U is the output (voltage, temperature etc. vector/matrix) 
% param is a structure containing other information, 
% for example param.PLS is a settings file (first part is strange) 
% 
% A typical use of the plw2ml function is as follows: 
% 
% [t,U]=plw2ml('c:\measure2\rotfungi\sl0921A.plw'); 
% plot(t,U(:,1)) 
% 
% A more advanced way is to use it on all files of a certain sort, 
% for example to plot the data in all files starting with 'sl' in one directory: 
% 
% presdir=cd; %present directory 
% dirvec=dir; %Find all filenames in directory 
% for k=1:length(dirvec) %go through all files 
% filename=dirvec(k).name; %put filename in filename-string 
% if length(filename)>5 % do not evaluate '-' '--' and other strange things in dirvec 
% if strcmp(filename(1:2),'SL') %only if it is a SLXXXX- file 
% fullfilename=[presdir,'\',filename]; %full filename, incl. path 
% [t,U]=PLW2ML(fullfilename); 
% plot(t,U); 
% end 
% end 
% end 
% 
% An even more automated evaluation method can be made by looking up, 
% e.g., sample identifiers in the param.PLS-variable (if they have 
% been entered into PLW before the start of the measurements). In the 
% following example samples (channels) are named HDMXX, where XX is a 
% sample number that the program finds and uses to sort the measurements 
% 
% k=k+1; 
% indstart=findstr('MultiConverter',param.PLS); %---Find line before channel identifier 
% indname=findstr('HDM',param.PLS(indstart:end)); %---In this example sample numbers follow HDM-string 
% %find all places in PLS-file starting with 'Name=HDM' (that are followed by the sample number) 
% for p=1:length(indname) %for all measurements found in the file 
% sample_no(k)=str2num(param.PLS(indstart+indname(p)+2:indstart+indname(p)+3)); %extract sample number 
% end 
% 
% Lars Wadsö, Building Materials, Lund University, Sweden 28 Oct 2004 

if nargout<2|nargout>3;error('PLW2ML needs two or three output arguments [t,U] or [t,U,param]');end 
if nargin==0;[fn,pn]=uigetfile('*.*','Open a measurement file');fullfilename=[pn,fn];info=0;end 
if nargin==1;info=0;end 
if fullfilename(end-3:end)~='.plw' & fullfilename(end-3:end)~='.PLW';error('PLW2ML can only open files with extension .plw or .PLW');end 
fid=fopen(fullfilename); %open file for reading 
if fid==-1;error('File could not be opened');end 
param.header_bytes=fread(fid,1,'ushort'); %read first line of PLW-HEADER (the end of the file contains a partial explanation of how a PLW file is built up) 
param.signature=char(fread(fid,40,'uchar'))'; %etc 
param.version=fread(fid,1,'uint32');
if info;disp(['PLW file version ',int2str(param.version)]);end 
switch param.version 
case 1 
antal_parametrar=50; 
antal_notes=200; 
case 2 
antal_parametrar=50; %100; 
antal_notes=200; 
case 3 
antal_parametrar=250; 
antal_notes=1000; 
case 4 
antal_parametrar=250; 
antal_notes=1000; 
end 
param.no_of_parameters=fread(fid,1,'uint32'); 
if info;disp(['Number of parameters ',int2str(param.no_of_parameters)]);end 
param.parameters=fread(fid,antal_parametrar,'uint16'); %says 50 in PLW manual 
if info;disp(['Parameters: ',int2str(param.parameters(1:param.no_of_parameters)')]);end 
% param.sample_no=0; 
% while param.sample_no==0 
param.sample_no=fread(fid,1,'uint32'); %=following 
% end 
if info;disp(['Number of samples: ',int2str(param.sample_no)]);end 
param.no_of_samples=fread(fid,1,'uint32'); %=previous %number of samples 
if info;disp(['No OF SAMPLES: ',int2str(param.no_of_samples)]);end 
param.max_samples=fread(fid,1,'uint32'); 
if info;disp(['MAX Number of samples: ',int2str(param.max_samples)]);end 
param.interval=fread(fid,1,'uint32'); %measurement interval 
if info;disp(['Interval: ',int2str(param.interval)]);end 
param.interval_units=fread(fid,1,'uint16'); %interval units 
if info;disp(['Interval_units: ',int2str(param.interval_units)]);end 
switch param.interval_units 
case 0;units=' fs'; 
case 1;units=' ps'; 
case 2;units=' ns'; 
case 3;units=' us'; 
case 4;units=' ms'; 
case 5;units=' s'; 
case 6;units=' min'; 
case 6;units=' h'; 
otherwise;units=' with unknown unit'; 
end 
if info;disp(['Sampling interval: ',num2str(param.interval),units]);end 
param.trigger_sample=fread(fid,1,'uint32'); 
param.triggered=fread(fid,1,'uint16'); 
param.first_sample=fread(fid,1,'uint32'); 
param.sample_bytes=fread(fid,1,'uint32'); 
param.settings_bytes=fread(fid,1,'uint32'); 
param.start_date=fread(fid,1,'uint32'); %start date (days since start of year 0) 
if info;disp(['Start date (days since 1 jan year 0) ',int2str(param.start_date)]);end 
param.start_time=fread(fid,1,'uint32'); %start time (seconds since start of day) 
if info;disp(['Start time (secondas since beginning of day) ',int2str(param.start_time)]);end 
param.minimum_time=fread(fid,1,'int32'); 
param.maximum_time=fread(fid,1,'int32'); 
param.notes=fread(fid,antal_notes,'uchar')'; 
param.current_time=fread(fid,1,'int32'); 
param.spare=fread(fid,78,'uint8'); 
%read DATA
t = zeros(1,param.no_of_samples);
U = zeros(1,param.no_of_samples);
for p=1:param.no_of_samples %here the samples are read 
t(p)=fread(fid,1,'uint')'; %time 
U(p)=fread(fid,param.no_of_parameters,'float'); %no_of_parameters of data 
end 
%read PLS-file appended at end of PLW-file (see the end of this file for an explanation) 
param.PLS=char(fread(fid,inf,'uchar'))'; %here the PLS-file is read (the first part contains strange text) 
if info;disp('The param.PLS file with infomation about the run can be retreived from a third output argument');end 
fclose(fid); %close file
end

Post Reply