Test and Measurement Forum

PLW file date format

Forum for discussing PicoLog

PLW file date format

Postby Jim Howes » Tue Jun 03, 2003 11:38 am

I am trying to grok the PLW file format as described on page 101 of the plw.pdf manual.

Two problems. The first I think I have worked out. The line
UNS16 parameters[50];

should, I think, read
UNS16 parameters[250];

or perhaps I am missing something else there? Either way nothing makes sense unless I sling an extra 200 bytes in there. There is also a highly mysterious gap between UNS32 start_time; and INT32 minimum_time; is something else missing?

Secondly, I am trying to work out how the start_date is encoded, without a lot of luck. The start time appears to be seconds past midnight, but I have not yet sussed the date format. Ideally, I need it as a C time_t

Regards,
Jim
Jim Howes
 

Postby markspencer » Wed Jun 04, 2003 12:58 pm

Hi,

Unfortunately, I am unable to provide the information you are requiring. The .plw file has changed and is likely to keep changing which means that the details in the help manual are out of date.

We are intending to rectify this in the near future, but I do not have a timescale for this.

Best regards,
Regards,

Mark Spencer
User avatar
markspencer
Site Admin
Site Admin
 
Posts: 598
Joined: Wed May 07, 2003 9:45 am

PLW files format

Postby dgibert » Wed Jun 04, 2003 1:07 pm

I also URGENTLY need to read binary PLW files on my Linux Computer. I agree that the users' guide concerning the PLW format is wrong.

Please, give us the right PLW format. A C or a MATLAB procedure for PLW reading would be welcome.

Thanks,

Dominique Gibert
dgibert
 

Postby markspencer » Thu Jun 05, 2003 7:07 am

Hi,

Unfortunately, I am unable to provide this information at present.

Best regards
Regards,

Mark Spencer
User avatar
markspencer
Site Admin
Site Admin
 
Posts: 598
Joined: Wed May 07, 2003 9:45 am

Re: PLW files format

Postby Jim Howes » Thu Jun 05, 2003 8:44 am

dgibert wrote:I also URGENTLY need to read binary PLW files on my Linux Computer. I agree that the users' guide concerning the PLW format is wrong.


As it happens, I've finished hacking around with my code and have something that works, atleast with the version of PLW files that my implementation creates.

My declaration reads:
Code: Select all
#pragma pack(1)  // Stupid alignment follows
typedef struct PICO_header_struct
{
   unsigned short   header_bytes;
   char      signature[40];
   unsigned long   version;
   unsigned long   no_of_parameters;
   unsigned short   parameters[250];   // !! The value in the PICOlog documentation is entirely bogus!
   unsigned long   sample_no;
   unsigned long   no_of_samples;
   unsigned long   max_samples;
   unsigned long   interval;
   unsigned short   interval_units;
   unsigned long   trigger_sample;
   unsigned short   triggered;
   unsigned long   first_sample;
   unsigned long   sample_bytes;
   unsigned long   settings_bytes;
   unsigned long   start_date;
   unsigned long   start_time;
   long      minimum_time;
   long      maximum_time;
   char      notes[200];
   long      current_time;
   unsigned char   spare[78];
} PICO_header_t;
#pragma pack()


Note that this is for MS Visual C, For GNU C, use #pragma align 1 if you have structure alignment problems, which I suspect you may have.

I process the date and time into a C time_t as follows
Code: Select all
   time_t   tBaseTime;
   struct tm *tmDate;
   PICO_header_t   PH;

   ...stuff to open file and fread((void *)&PH,sizeof(PH),1,inputfile)...

   tBaseTime = 86400 * (PH.start_date - 719163) + PH.start_time;
   tmDate = gmtime(&tBaseTime);

Note I've used gmtime(). However, I think this might be wrong. Windows applications are notoriously bad at handling time zones properly in my opinion; comes from a history of assuming that your user is sat infront of the PC all the time; a luxury us unix hackers have never had to put up with. Quite what happens when your log file includes times during the changeover from/to DST, I don't know. localtime() is another function you could try.

Before reading data, I have to
Code: Select all
   fseek(input, PH.header_bytes, SEEK_SET);

because sizeof(PH) appears to bear absolutely no relation to the actual layout of the file because the documentation is entirely bogus in that regard. The suspicious gap between start_time and minimum_time in the documentation suggest that the whole story isn't being told.

Anyway, make of the above what you will. It comes with no warranty, and if it breaks you get to keep both pieces.

Regards,
Jim <jimhowes at prosig dot com>
Jim Howes
 

Postby dgibert » Fri Jun 06, 2003 7:55 am

Thank you very much Jim,

indeed the Pico indications are quite false ! I'll try your code on my files.

Beyond this, I am surprised by the Moderator response. How is it possible to ignore the PLW format ? Does my picolog software version write PLW files without knowing the exact file format. I am EFFRAID !!!

Regards and thanks again,

Dominique
dgibert
 

plw file format

Postby P. F. R. » Fri Apr 16, 2004 8:45 am

hi all,

I've worked out the plw file format. here is it:
unsigned integer, 16 bit: uint16
unsigned integer, 32 bit: uint32

uint16 header size
char[40] signature
uint32 version
uint32 no. of parameters
uint16[250] parameters
uint32 sample no.
uint32 sample no. (twice)
uint32 max sample
uint32 interval
uint16 interval units
uint32 trigger sample
uint16 triggered
uint32 first sample
uint32 sample byte
uint32 setting byte
uint32 start date
uint32 start time
uint32 min time
uint32 max time
char[1000] notes
uint32 current time
uint8[78] spare

that's it! it gives a total header size of 1684 bytes, which seems to be alright.note that the informations given in the manual are wrong on the size of the array "parameters" and "notes".
hope this will be useful...
cheers!
Pierre-François
P. F. R.
 

start_time

Postby P.F.R. » Thu Jun 24, 2004 7:22 am

Hi all,
I've also found that the start_time parameter was simply...the number of seconds elapsed in one day (starting at midnight).
sounds simple, isn't it? :D
Bless bless
P.F.R.
 

PLW DATA FILE

Postby TK » Wed Nov 03, 2004 3:09 pm

hi
could some one help me please. I need to know exactly what PLW files are, how to read and use them. I have not been able to understand the user manul of PICO LOG. please e-mail me at teekay_tk@hotmail.com
i shall be grateful
regards

taimur
TK
 

Postby markspencer » Fri Nov 05, 2004 8:24 am

Hi,

I am sorry to hear that you are experiencing problems with understanding the layout for the PLW files. Unfortunately, the help manual is not totaly up to date on the structure used, but if you look at P.F.R post of 16 Apr 04, this gives the correct format. Now that you know this you can use this structure to read the byes out of the file.

Best regards,
Regards,

Mark Spencer
User avatar
markspencer
Site Admin
Site Admin
 
Posts: 598
Joined: Wed May 07, 2003 9:45 am

Postby leepatrk » Sun Mar 19, 2006 1:18 am

Any updates on this? I tried the format above by P.F.R and it doesn't quite align. I'm using the latest version of Picolog: 5.13.7

Is the order of data still the same?

THanks,
Lee
leepatrk
Newbie
 
Posts: 1
Joined: Sun Mar 19, 2006 12:47 am

Postby Sarah » Fri Mar 24, 2006 11:56 am

Hi Lee

No I believe the file format has changed since that post was written. Does the help manual provide you with the information that you require?

Best Regards
Sarah
 

Postby LarsW » Sat Mar 25, 2006 10:34 pm

Some time ago I wrote a MATLAB-program to read PLW-files directly into MATLAB. I remember that the manual did not give the whole truth, but also that (if I remember correctly) different versions (I found three) were slightly different. So if you want a program that works for all PLW-files you need to check for "version" have slightly different code for different version.
LarsW
Active User
Active User
 
Posts: 6
Joined: Thu Mar 23, 2006 9:04 am
Location: Lund University

Postby LarsW » Mon Mar 27, 2006 6:30 am

Here is my program PLW2ML.m that converts PLW-files to MATLAB variables. On lines 59-69 you can see the differences I noted between the different versions 1, 2 and 3. Note that the param.PLS, which contains information about the run (for example the names given to the channels) had an initial part that I could not decipher. I wrote this program ago year and a half ago and have used it to evaluate files in an automated way. Look at the examples in the initial comments (starting with %). I especially liked the the way I evaluated a large number of PLW-files, each with measurements on eight randomly chosen samples, and then found which sample that had been in each channel in each run by looking that up in param.PLS (In each run I named the channels HDMXX, with XX being the number of the sample).

Yesterday I tested the program on some files of version 2 and 3 I found in my computer. For many files it worked well and was quick, but for some version 2 files the reading of the data (the loop in lines 114-117) took several minutes even for not too large data files. It could probably be improved by using some more efficient way of reading the data matrix into MATLAB. So I guess the MATLAB code is not optimal AND it may even not be correct for all types of PLW-files...(is there a version 4?).

Here is the code (the Picotech Forum would not allow me to attach an m-file, so I had to paste it into the message)

function [t,U,param]=PLW2ML(fullfilename,info)
% PLW2ML converts PLW-files (PicoTech Picologger) to MATLAB variables
% [t,U,param]=PLW2ML(filename,info)
% filename is the full filename (with path and extension .plw)
% info is an optional parameter that if set to 1 displays
% information about the file being read
% t is the time vector
% U is the output (voltage, temperature etc. vector/matrix)
% param is a structure containing other information,
% for example param.PLS is a settings file (first part is strange)
%
% A typical use of the plw2ml function is as follows:
%
% [t,U]=plw2ml('c:\measure2\rotfungi\sl0921A.plw');
% plot(t,U(:,1))
%
% A more advanced way is to use it on all files of a certain sort,
% for example to plot the data in all files starting with 'sl' in one directory:
%
% presdir=cd; %present directory
% dirvec=dir; %Find all filenames in directory
% for k=1:length(dirvec) %go through all files
% filename=dirvec(k).name; %put filename in filename-string
% if length(filename)>5 % do not evaluate '-' '--' and other strange things in dirvec
% if strcmp(filename(1:2),'SL') %only if it is a SLXXXX- file
% fullfilename=[presdir,'\',filename]; %full filename, incl. path
% [t,U]=PLW2ML(fullfilename);
% plot(t,U);
% end
% end
% end
%
% An even more automated evaluation method can be made by looking up,
% e.g., sample identifiers in the param.PLS-variable (if they have
% been entered into PLW before the start of the measurements). In the
% following example samples (channels) are named HDMXX, where XX is a
% sample number that the program finds and uses to sort the measurements
%
% k=k+1;
% indstart=findstr('MultiConverter',param.PLS); %---Find line before channel identifier
% indname=findstr('HDM',param.PLS(indstart:end)); %---In this example sample numbers follow HDM-string
% %find all places in PLS-file starting with 'Name=HDM' (that are followed by the sample number)
% for p=1:length(indname) %for all measurements found in the file
% sample_no(k)=str2num(param.PLS(indstart+indname(p)+2:indstart+indname(p)+3)); %extract sample number
% end
%
% Lars Wadsö, Building Materials, Lund University, Sweden 28 Oct 2004

if nargout<2|nargout>3;error('PLW2ML needs two or three output arguments [t,U] or [t,U,param]');end
if nargin==0;[fn,pn]=uigetfile('*.*','Open a measurement file');fullfilename=[pn,fn];info=0;end
if nargin==1;info=0;end
if fullfilename(end-3:end)~='.plw' & fullfilename(end-3:end)~='.PLW';error('PLW2ML can only open files with extension .plw or .PLW');end
fid=fopen(fullfilename); %open file for reading
if fid==-1;error('File could not be opened');end
param.header_bytes=fread(fid,1,'ushort'); %read first line of PLW-HEADER (the end of the file contains a partial explanation of how a PLW file is built up)
param.signature=char(fread(fid,40,'uchar'))'; %etc
param.version=fread(fid,1,'uint32');
if info;disp(['PLW file version ',int2str(param.version)]);end
switch param.version
case 1
antal_parametrar=50;
antal_notes=200;
case 2
antal_parametrar=50; %100;
antal_notes=200;
case 3
antal_parametrar=250;
antal_notes=1000;
end
param.no_of_parameters=fread(fid,1,'uint32');
if info;disp(['Number of parameters ',int2str(param.no_of_parameters)]);end
param.parameters=fread(fid,antal_parametrar,'uint16'); %says 50 in PLW manual
if info;disp(['Parameters: ',int2str(param.parameters(1:param.no_of_parameters)')]);end
% param.sample_no=0;
% while param.sample_no==0
param.sample_no=fread(fid,1,'uint32'); %=following
% end
if info;disp(['Number of samples: ',int2str(param.sample_no)]);end
param.no_of_samples=fread(fid,1,'uint32'); %=previous %number of samples
if info;disp(['No OF SAMPLES: ',int2str(param.no_of_samples)]);end
param.max_samples=fread(fid,1,'uint32');
if info;disp(['MAX Number of samples: ',int2str(param.max_samples)]);end
param.interval=fread(fid,1,'uint32'); %measurement interval
if info;disp(['Interval: ',int2str(param.interval)]);end
param.interval_units=fread(fid,1,'uint16'); %interval units
if info;disp(['Interval_units: ',int2str(param.interval_units)]);end
switch param.interval_units
case 0;units=' fs';
case 1;units=' ps';
case 2;units=' ns';
case 3;units=' us';
case 4;units=' ms';
case 5;units=' s';
case 6;units=' min';
case 6;units=' h';
otherwise;units=' with unknown unit';
end
if info;disp(['Sampling interval: ',num2str(param.interval),units]);end
param.trigger_sample=fread(fid,1,'uint32');
param.triggered=fread(fid,1,'uint16');
param.first_sample=fread(fid,1,'uint32');
param.sample_bytes=fread(fid,1,'uint32');
param.settings_bytes=fread(fid,1,'uint32');
param.start_date=fread(fid,1,'uint32'); %start date (days since start of year 0)
if info;disp(['Start date (days since 1 jan year 0) ',int2str(param.start_date)]);end
param.start_time=fread(fid,1,'uint32'); %start time (seconds since start of day)
if info;disp(['Start time (secondas since beginning of day) ',int2str(param.start_time)]);end
param.minimum_time=fread(fid,1,'int32');
param.maximum_time=fread(fid,1,'int32');
param.notes=fread(fid,antal_notes,'uchar')';
param.current_time=fread(fid,1,'int32');
param.spare=fread(fid,78,'uint8');
%read DATA
for p=1:param.no_of_samples %here the samples are read
t(p)=fread(fid,1,'uint')'; %time
U(p,:)=fread(fid,param.no_of_parameters,'float'); %no_of_parameters of data
end
%read PLS-file appended at end of PLW-file (see the end of this file for an explanation)
param.PLS=char(fread(fid,inf,'uchar'))'; %here the PLS-file is read (the first part contains strange text)
if info;disp('The param.PLS file with infomation about the run can be retreived from a third output argument');end
fclose(fid); %close file
LarsW
Active User
Active User
 
Posts: 6
Joined: Thu Mar 23, 2006 9:04 am
Location: Lund University

Postby Guest » Sat May 20, 2006 10:04 am

Thanks for the code Lars, i made some improvements for it;

- Support for v4
- Increased speed !!

Code: Select all
function [t,U,param]=PLW2ML(fullfilename,info)
% PLW2ML converts PLW-files (PicoTech Picologger) to MATLAB variables
% [t,U,param]=PLW2ML(filename,info)
% filename is the full filename (with path and extension .plw)
% info is an optional parameter that if set to 1 displays
% information about the file being read
% t is the time vector
% U is the output (voltage, temperature etc. vector/matrix)
% param is a structure containing other information,
% for example param.PLS is a settings file (first part is strange)
%
% A typical use of the plw2ml function is as follows:
%
% [t,U]=plw2ml('c:\measure2\rotfungi\sl0921A.plw');
% plot(t,U(:,1))
%
% A more advanced way is to use it on all files of a certain sort,
% for example to plot the data in all files starting with 'sl' in one directory:
%
% presdir=cd; %present directory
% dirvec=dir; %Find all filenames in directory
% for k=1:length(dirvec) %go through all files
% filename=dirvec(k).name; %put filename in filename-string
% if length(filename)>5 % do not evaluate '-' '--' and other strange things in dirvec
% if strcmp(filename(1:2),'SL') %only if it is a SLXXXX- file
% fullfilename=[presdir,'\',filename]; %full filename, incl. path
% [t,U]=PLW2ML(fullfilename);
% plot(t,U);
% end
% end
% end
%
% An even more automated evaluation method can be made by looking up,
% e.g., sample identifiers in the param.PLS-variable (if they have
% been entered into PLW before the start of the measurements). In the
% following example samples (channels) are named HDMXX, where XX is a
% sample number that the program finds and uses to sort the measurements
%
% k=k+1;
% indstart=findstr('MultiConverter',param.PLS); %---Find line before channel identifier
% indname=findstr('HDM',param.PLS(indstart:end)); %---In this example sample numbers follow HDM-string
% %find all places in PLS-file starting with 'Name=HDM' (that are followed by the sample number)
% for p=1:length(indname) %for all measurements found in the file
% sample_no(k)=str2num(param.PLS(indstart+indname(p)+2:indstart+indname(p)+3)); %extract sample number
% end
%
% Lars Wadsö, Building Materials, Lund University, Sweden 28 Oct 2004

if nargout<2|nargout>3;error('PLW2ML needs two or three output arguments [t,U] or [t,U,param]');end
if nargin==0;[fn,pn]=uigetfile('*.*','Open a measurement file');fullfilename=[pn,fn];info=0;end
if nargin==1;info=0;end
if fullfilename(end-3:end)~='.plw' & fullfilename(end-3:end)~='.PLW';error('PLW2ML can only open files with extension .plw or .PLW');end
fid=fopen(fullfilename); %open file for reading
if fid==-1;error('File could not be opened');end
param.header_bytes=fread(fid,1,'ushort'); %read first line of PLW-HEADER (the end of the file contains a partial explanation of how a PLW file is built up)
param.signature=char(fread(fid,40,'uchar'))'; %etc
param.version=fread(fid,1,'uint32');
if info;disp(['PLW file version ',int2str(param.version)]);end
switch param.version
case 1
antal_parametrar=50;
antal_notes=200;
case 2
antal_parametrar=50; %100;
antal_notes=200;
case 3
antal_parametrar=250;
antal_notes=1000;
case 4
antal_parametrar=250;
antal_notes=1000;
end
param.no_of_parameters=fread(fid,1,'uint32');
if info;disp(['Number of parameters ',int2str(param.no_of_parameters)]);end
param.parameters=fread(fid,antal_parametrar,'uint16'); %says 50 in PLW manual
if info;disp(['Parameters: ',int2str(param.parameters(1:param.no_of_parameters)')]);end
% param.sample_no=0;
% while param.sample_no==0
param.sample_no=fread(fid,1,'uint32'); %=following
% end
if info;disp(['Number of samples: ',int2str(param.sample_no)]);end
param.no_of_samples=fread(fid,1,'uint32'); %=previous %number of samples
if info;disp(['No OF SAMPLES: ',int2str(param.no_of_samples)]);end
param.max_samples=fread(fid,1,'uint32');
if info;disp(['MAX Number of samples: ',int2str(param.max_samples)]);end
param.interval=fread(fid,1,'uint32'); %measurement interval
if info;disp(['Interval: ',int2str(param.interval)]);end
param.interval_units=fread(fid,1,'uint16'); %interval units
if info;disp(['Interval_units: ',int2str(param.interval_units)]);end
switch param.interval_units
case 0;units=' fs';
case 1;units=' ps';
case 2;units=' ns';
case 3;units=' us';
case 4;units=' ms';
case 5;units=' s';
case 6;units=' min';
case 6;units=' h';
otherwise;units=' with unknown unit';
end
if info;disp(['Sampling interval: ',num2str(param.interval),units]);end
param.trigger_sample=fread(fid,1,'uint32');
param.triggered=fread(fid,1,'uint16');
param.first_sample=fread(fid,1,'uint32');
param.sample_bytes=fread(fid,1,'uint32');
param.settings_bytes=fread(fid,1,'uint32');
param.start_date=fread(fid,1,'uint32'); %start date (days since start of year 0)
if info;disp(['Start date (days since 1 jan year 0) ',int2str(param.start_date)]);end
param.start_time=fread(fid,1,'uint32'); %start time (seconds since start of day)
if info;disp(['Start time (secondas since beginning of day) ',int2str(param.start_time)]);end
param.minimum_time=fread(fid,1,'int32');
param.maximum_time=fread(fid,1,'int32');
param.notes=fread(fid,antal_notes,'uchar')';
param.current_time=fread(fid,1,'int32');
param.spare=fread(fid,78,'uint8');
%read DATA
t = zeros(1,param.no_of_samples);
U = zeros(1,param.no_of_samples);
for p=1:param.no_of_samples %here the samples are read
t(p)=fread(fid,1,'uint')'; %time
U(p)=fread(fid,param.no_of_parameters,'float'); %no_of_parameters of data
end
%read PLS-file appended at end of PLW-file (see the end of this file for an explanation)
param.PLS=char(fread(fid,inf,'uchar'))'; %here the PLS-file is read (the first part contains strange text)
if info;disp('The param.PLS file with infomation about the run can be retreived from a third output argument');end
fclose(fid); %close file
end
Guest
 

Next

Return to PicoLog

Who is online

Users browsing this forum: No registered users and 1 guest