PLW file date format
Goto page 1, 2  Next
 
Post new topic   Reply to topic    Pico Technology Forum Index -> PicoLog
Author Message
Jim Howes
Guest





PostPosted: Tue Jun 03, 2003 11:38 am    Post subject: PLW file date format Reply with quote

I am trying to grok the PLW file format as described on page 101 of the plw.pdf manual.

Two problems. The first I think I have worked out. The line
UNS16 parameters[50];

should, I think, read
UNS16 parameters[250];

or perhaps I am missing something else there? Either way nothing makes sense unless I sling an extra 200 bytes in there. There is also a highly mysterious gap between UNS32 start_time; and INT32 minimum_time; is something else missing?

Secondly, I am trying to work out how the start_date is encoded, without a lot of luck. The start time appears to be seconds past midnight, but I have not yet sussed the date format. Ideally, I need it as a C time_t

Regards,
Jim
Back to top
markspencer
Site Admin
Site Admin


Joined: 07 May 2003
Posts: 610

PostPosted: Wed Jun 04, 2003 12:58 pm    Post subject: Reply with quote

Hi,

Unfortunately, I am unable to provide the information you are requiring. The .plw file has changed and is likely to keep changing which means that the details in the help manual are out of date.

We are intending to rectify this in the near future, but I do not have a timescale for this.

Best regards,
Back to top
dgibert
Guest





PostPosted: Wed Jun 04, 2003 1:07 pm    Post subject: PLW files format Reply with quote

I also URGENTLY need to read binary PLW files on my Linux Computer. I agree that the users' guide concerning the PLW format is wrong.

Please, give us the right PLW format. A C or a MATLAB procedure for PLW reading would be welcome.

Thanks,

Dominique Gibert
Back to top
markspencer
Site Admin
Site Admin


Joined: 07 May 2003
Posts: 610

PostPosted: Thu Jun 05, 2003 7:07 am    Post subject: Reply with quote

Hi,

Unfortunately, I am unable to provide this information at present.

Best regards
Back to top
Jim Howes
Guest





PostPosted: Thu Jun 05, 2003 8:44 am    Post subject: Re: PLW files format Reply with quote

dgibert wrote:
I also URGENTLY need to read binary PLW files on my Linux Computer. I agree that the users' guide concerning the PLW format is wrong.


As it happens, I've finished hacking around with my code and have something that works, atleast with the version of PLW files that my implementation creates.

My declaration reads:
Code:

#pragma pack(1)  // Stupid alignment follows
typedef struct PICO_header_struct
{
   unsigned short   header_bytes;
   char      signature[40];
   unsigned long   version;
   unsigned long   no_of_parameters;
   unsigned short   parameters[250];   // !! The value in the PICOlog documentation is entirely bogus!
   unsigned long   sample_no;
   unsigned long   no_of_samples;
   unsigned long   max_samples;
   unsigned long   interval;
   unsigned short   interval_units;
   unsigned long   trigger_sample;
   unsigned short   triggered;
   unsigned long   first_sample;
   unsigned long   sample_bytes;
   unsigned long   settings_bytes;
   unsigned long   start_date;
   unsigned long   start_time;
   long      minimum_time;
   long      maximum_time;
   char      notes[200];
   long      current_time;
   unsigned char   spare[78];
} PICO_header_t;
#pragma pack()


Note that this is for MS Visual C, For GNU C, use #pragma align 1 if you have structure alignment problems, which I suspect you may have.

I process the date and time into a C time_t as follows
Code:

   time_t   tBaseTime;
   struct tm *tmDate;
   PICO_header_t   PH;

   ...stuff to open file and fread((void *)&PH,sizeof(PH),1,inputfile)...

   tBaseTime = 86400 * (PH.start_date - 719163) + PH.start_time;
   tmDate = gmtime(&tBaseTime);

Note I've used gmtime(). However, I think this might be wrong. Windows applications are notoriously bad at handling time zones properly in my opinion; comes from a history of assuming that your user is sat infront of the PC all the time; a luxury us unix hackers have never had to put up with. Quite what happens when your log file includes times during the changeover from/to DST, I don't know. localtime() is another function you could try.

Before reading data, I have to
Code:

   fseek(input, PH.header_bytes, SEEK_SET);

because sizeof(PH) appears to bear absolutely no relation to the actual layout of the file because the documentation is entirely bogus in that regard. The suspicious gap between start_time and minimum_time in the documentation suggest that the whole story isn't being told.

Anyway, make of the above what you will. It comes with no warranty, and if it breaks you get to keep both pieces.

Regards,
Jim <jimhowes at prosig dot com>
Back to top
dgibert
Guest





PostPosted: Fri Jun 06, 2003 7:55 am    Post subject: Reply with quote

Thank you very much Jim,

indeed the Pico indications are quite false ! I'll try your code on my files.

Beyond this, I am surprised by the Moderator response. How is it possible to ignore the PLW format ? Does my picolog software version write PLW files without knowing the exact file format. I am EFFRAID !!!

Regards and thanks again,

Dominique
Back to top
P. F. R.
Guest





PostPosted: Fri Apr 16, 2004 8:45 am    Post subject: plw file format Reply with quote

hi all,

I've worked out the plw file format. here is it:
unsigned integer, 16 bit: uint16
unsigned integer, 32 bit: uint32

uint16 header size
char[40] signature
uint32 version
uint32 no. of parameters
uint16[250] parameters
uint32 sample no.
uint32 sample no. (twice)
uint32 max sample
uint32 interval
uint16 interval units
uint32 trigger sample
uint16 triggered
uint32 first sample
uint32 sample byte
uint32 setting byte
uint32 start date
uint32 start time
uint32 min time
uint32 max time
char[1000] notes
uint32 current time
uint8[78] spare

that's it! it gives a total header size of 1684 bytes, which seems to be alright.note that the informations given in the manual are wrong on the size of the array "parameters" and "notes".
hope this will be useful...
cheers!
Pierre-François
Back to top
P.F.R.
Guest





PostPosted: Thu Jun 24, 2004 7:22 am    Post subject: start_time Reply with quote

Hi all,
I've also found that the start_time parameter was simply...the number of seconds elapsed in one day (starting at midnight).
sounds simple, isn't it? Very Happy
Bless bless
Back to top
TK
Guest





PostPosted: Wed Nov 03, 2004 3:09 pm    Post subject: PLW DATA FILE Reply with quote

hi
could some one help me please. I need to know exactly what PLW files are, how to read and use them. I have not been able to understand the user manul of PICO LOG. please e-mail me at teekay_tk@hotmail.com
i shall be grateful
regards

taimur
Back to top
markspencer
Site Admin
Site Admin


Joined: 07 May 2003
Posts: 610

PostPosted: Fri Nov 05, 2004 8:24 am    Post subject: Reply with quote

Hi,

I am sorry to hear that you are experiencing problems with understanding the layout for the PLW files. Unfortunately, the help manual is not totaly up to date on the structure used, but if you look at P.F.R post of 16 Apr 04, this gives the correct format. Now that you know this you can use this structure to read the byes out of the file.

Best regards,
Back to top
leepatrk
Newbie


Joined: 19 Mar 2006
Posts: 1

PostPosted: Sun Mar 19, 2006 1:18 am    Post subject: Reply with quote

Any updates on this? I tried the format above by P.F.R and it doesn't quite align. I'm using the latest version of Picolog: 5.13.7

Is the order of data still the same?

THanks,
Lee
Back to top
Sarah
Guest





PostPosted: Fri Mar 24, 2006 11:56 am    Post subject: Reply with quote

Hi Lee

No I believe the file format has changed since that post was written. Does the help manual provide you with the information that you require?

Best Regards
Back to top
LarsW
Active User
Active User


Joined: 23 Mar 2006
Posts: 6
Location: Lund University

PostPosted: Sat Mar 25, 2006 10:34 pm    Post subject: Reply with quote

Some time ago I wrote a MATLAB-program to read PLW-files directly into MATLAB. I remember that the manual did not give the whole truth, but also that (if I remember correctly) different versions (I found three) were slightly different. So if you want a program that works for all PLW-files you need to check for "version" have slightly different code for different version.
Back to top
LarsW
Active User
Active User


Joined: 23 Mar 2006
Posts: 6
Location: Lund University

PostPosted: Mon Mar 27, 2006 6:30 am    Post subject: Reply with quote

Here is my program PLW2ML.m that converts PLW-files to MATLAB variables. On lines 59-69 you can see the differences I noted between the different versions 1, 2 and 3. Note that the param.PLS, which contains information about the run (for example the names given to the channels) had an initial part that I could not decipher. I wrote this program ago year and a half ago and have used it to evaluate files in an automated way. Look at the examples in the initial comments (starting with %). I especially liked the the way I evaluated a large number of PLW-files, each with measurements on eight randomly chosen samples, and then found which sample that had been in each channel in each run by looking that up in param.PLS (In each run I named the channels HDMXX, with XX being the number of the sample).

Yesterday I tested the program on some files of version 2 and 3 I found in my computer. For many files it worked well and was quick, but for some version 2 files the reading of the data (the loop in lines 114-117) took several minutes even for not too large data files. It could probably be improved by using some more efficient way of reading the data matrix into MATLAB. So I guess the MATLAB code is not optimal AND it may even not be correct for all types of PLW-files...(is there a version 4?).

Here is the code (the Picotech Forum would not allow me to attach an m-file, so I had to paste it into the message)

function [t,U,param]=PLW2ML(fullfilename,info)
% PLW2ML converts PLW-files (PicoTech Picologger) to MATLAB variables
% [t,U,param]=PLW2ML(filename,info)
% filename is the full filename (with path and extension .plw)
% info is an optional parameter that if set to 1 displays
% information about the file being read
% t is the time vector
% U is the output (voltage, temperature etc. vector/matrix)
% param is a structure containing other information,
% for example param.PLS is a settings file (first part is strange)
%
% A typical use of the plw2ml function is as follows:
%
% [t,U]=plw2ml('c:\measure2\rotfungi\sl0921A.plw');
% plot(t,U(:,1))
%
% A more advanced way is to use it on all files of a certain sort,
% for example to plot the data in all files starting with 'sl' in one directory:
%
% presdir=cd; %present directory
% dirvec=dir; %Find all filenames in directory
% for k=1:length(dirvec) %go through all files
% filename=dirvec(k).name; %put filename in filename-string
% if length(filename)>5 % do not evaluate '-' '--' and other strange things in dirvec
% if strcmp(filename(1:2),'SL') %only if it is a SLXXXX- file
% fullfilename=[presdir,'\',filename]; %full filename, incl. path
% [t,U]=PLW2ML(fullfilename);
% plot(t,U);
% end
% end
% end
%
% An even more automated evaluation method can be made by looking up,
% e.g., sample identifiers in the param.PLS-variable (if they have
% been entered into PLW before the start of the measurements). In the
% following example samples (channels) are named HDMXX, where XX is a
% sample number that the program finds and uses to sort the measurements
%
% k=k+1;
% indstart=findstr('MultiConverter',param.PLS); %---Find line before channel identifier
% indname=findstr('HDM',param.PLS(indstart:end)); %---In this example sample numbers follow HDM-string
% %find all places in PLS-file starting with 'Name=HDM' (that are followed by the sample number)
% for p=1:length(indname) %for all measurements found in the file
% sample_no(k)=str2num(param.PLS(indstart+indname(p)+2:indstart+indname(p)+3)); %extract sample number
% end
%
% Lars Wadsö, Building Materials, Lund University, Sweden 28 Oct 2004

if nargout<2|nargout>3;error('PLW2ML needs two or three output arguments [t,U] or [t,U,param]');end
if nargin==0;[fn,pn]=uigetfile('*.*','Open a measurement file');fullfilename=[pn,fn];info=0;end
if nargin==1;info=0;end
if fullfilename(end-3:end)~='.plw' & fullfilename(end-3:end)~='.PLW';error('PLW2ML can only open files with extension .plw or .PLW');end
fid=fopen(fullfilename); %open file for reading
if fid==-1;error('File could not be opened');end
param.header_bytes=fread(fid,1,'ushort'); %read first line of PLW-HEADER (the end of the file contains a partial explanation of how a PLW file is built up)
param.signature=char(fread(fid,40,'uchar'))'; %etc
param.version=fread(fid,1,'uint32');
if info;disp(['PLW file version ',int2str(param.version)]);end
switch param.version
case 1
antal_parametrar=50;
antal_notes=200;
case 2
antal_parametrar=50; %100;
antal_notes=200;
case 3
antal_parametrar=250;
antal_notes=1000;
end
param.no_of_parameters=fread(fid,1,'uint32');
if info;disp(['Number of parameters ',int2str(param.no_of_parameters)]);end
param.parameters=fread(fid,antal_parametrar,'uint16'); %says 50 in PLW manual
if info;disp(['Parameters: ',int2str(param.parameters(1:param.no_of_parameters)')]);end
% param.sample_no=0;
% while param.sample_no==0
param.sample_no=fread(fid,1,'uint32'); %=following
% end
if info;disp(['Number of samples: ',int2str(param.sample_no)]);end
param.no_of_samples=fread(fid,1,'uint32'); %=previous %number of samples
if info;disp(['No OF SAMPLES: ',int2str(param.no_of_samples)]);end
param.max_samples=fread(fid,1,'uint32');
if info;disp(['MAX Number of samples: ',int2str(param.max_samples)]);end
param.interval=fread(fid,1,'uint32'); %measurement interval
if info;disp(['Interval: ',int2str(param.interval)]);end
param.interval_units=fread(fid,1,'uint16'); %interval units
if info;disp(['Interval_units: ',int2str(param.interval_units)]);end
switch param.interval_units
case 0;units=' fs';
case 1;units=' ps';
case 2;units=' ns';
case 3;units=' us';
case 4;units=' ms';
case 5;units=' s';
case 6;units=' min';
case 6;units=' h';
otherwise;units=' with unknown unit';
end
if info;disp(['Sampling interval: ',num2str(param.interval),units]);end
param.trigger_sample=fread(fid,1,'uint32');
param.triggered=fread(fid,1,'uint16');
param.first_sample=fread(fid,1,'uint32');
param.sample_bytes=fread(fid,1,'uint32');
param.settings_bytes=fread(fid,1,'uint32');
param.start_date=fread(fid,1,'uint32'); %start date (days since start of year 0)
if info;disp(['Start date (days since 1 jan year 0) ',int2str(param.start_date)]);end
param.start_time=fread(fid,1,'uint32'); %start time (seconds since start of day)
if info;disp(['Start time (secondas since beginning of day) ',int2str(param.start_time)]);end
param.minimum_time=fread(fid,1,'int32');
param.maximum_time=fread(fid,1,'int32');
param.notes=fread(fid,antal_notes,'uchar')';
param.current_time=fread(fid,1,'int32');
param.spare=fread(fid,78,'uint8');
%read DATA
for p=1:param.no_of_samples %here the samples are read
t(p)=fread(fid,1,'uint')'; %time
U(p,:)=fread(fid,param.no_of_parameters,'float'); %no_of_parameters of data
end
%read PLS-file appended at end of PLW-file (see the end of this file for an explanation)
param.PLS=char(fread(fid,inf,'uchar'))'; %here the PLS-file is read (the first part contains strange text)
if info;disp('The param.PLS file with infomation about the run can be retreived from a third output argument');end
fclose(fid); %close file
Back to top
Guest






PostPosted: Sat May 20, 2006 10:04 am    Post subject: Reply with quote

Thanks for the code Lars, i made some improvements for it;

- Support for v4
- Increased speed !!

Code:

function [t,U,param]=PLW2ML(fullfilename,info)
% PLW2ML converts PLW-files (PicoTech Picologger) to MATLAB variables
% [t,U,param]=PLW2ML(filename,info)
% filename is the full filename (with path and extension .plw)
% info is an optional parameter that if set to 1 displays
% information about the file being read
% t is the time vector
% U is the output (voltage, temperature etc. vector/matrix)
% param is a structure containing other information,
% for example param.PLS is a settings file (first part is strange)
%
% A typical use of the plw2ml function is as follows:
%
% [t,U]=plw2ml('c:\measure2\rotfungi\sl0921A.plw');
% plot(t,U(:,1))
%
% A more advanced way is to use it on all files of a certain sort,
% for example to plot the data in all files starting with 'sl' in one directory:
%
% presdir=cd; %present directory
% dirvec=dir; %Find all filenames in directory
% for k=1:length(dirvec) %go through all files
% filename=dirvec(k).name; %put filename in filename-string
% if length(filename)>5 % do not evaluate '-' '--' and other strange things in dirvec
% if strcmp(filename(1:2),'SL') %only if it is a SLXXXX- file
% fullfilename=[presdir,'\',filename]; %full filename, incl. path
% [t,U]=PLW2ML(fullfilename);
% plot(t,U);
% end
% end
% end
%
% An even more automated evaluation method can be made by looking up,
% e.g., sample identifiers in the param.PLS-variable (if they have
% been entered into PLW before the start of the measurements). In the
% following example samples (channels) are named HDMXX, where XX is a
% sample number that the program finds and uses to sort the measurements
%
% k=k+1;
% indstart=findstr('MultiConverter',param.PLS); %---Find line before channel identifier
% indname=findstr('HDM',param.PLS(indstart:end)); %---In this example sample numbers follow HDM-string
% %find all places in PLS-file starting with 'Name=HDM' (that are followed by the sample number)
% for p=1:length(indname) %for all measurements found in the file
% sample_no(k)=str2num(param.PLS(indstart+indname(p)+2:indstart+indname(p)+3)); %extract sample number
% end
%
% Lars Wadsö, Building Materials, Lund University, Sweden 28 Oct 2004

if nargout<2|nargout>3;error('PLW2ML needs two or three output arguments [t,U] or [t,U,param]');end
if nargin==0;[fn,pn]=uigetfile('*.*','Open a measurement file');fullfilename=[pn,fn];info=0;end
if nargin==1;info=0;end
if fullfilename(end-3:end)~='.plw' & fullfilename(end-3:end)~='.PLW';error('PLW2ML can only open files with extension .plw or .PLW');end
fid=fopen(fullfilename); %open file for reading
if fid==-1;error('File could not be opened');end
param.header_bytes=fread(fid,1,'ushort'); %read first line of PLW-HEADER (the end of the file contains a partial explanation of how a PLW file is built up)
param.signature=char(fread(fid,40,'uchar'))'; %etc
param.version=fread(fid,1,'uint32');
if info;disp(['PLW file version ',int2str(param.version)]);end
switch param.version
case 1
antal_parametrar=50;
antal_notes=200;
case 2
antal_parametrar=50; %100;
antal_notes=200;
case 3
antal_parametrar=250;
antal_notes=1000;
case 4
antal_parametrar=250;
antal_notes=1000;
end
param.no_of_parameters=fread(fid,1,'uint32');
if info;disp(['Number of parameters ',int2str(param.no_of_parameters)]);end
param.parameters=fread(fid,antal_parametrar,'uint16'); %says 50 in PLW manual
if info;disp(['Parameters: ',int2str(param.parameters(1:param.no_of_parameters)')]);end
% param.sample_no=0;
% while param.sample_no==0
param.sample_no=fread(fid,1,'uint32'); %=following
% end
if info;disp(['Number of samples: ',int2str(param.sample_no)]);end
param.no_of_samples=fread(fid,1,'uint32'); %=previous %number of samples
if info;disp(['No OF SAMPLES: ',int2str(param.no_of_samples)]);end
param.max_samples=fread(fid,1,'uint32');
if info;disp(['MAX Number of samples: ',int2str(param.max_samples)]);end
param.interval=fread(fid,1,'uint32'); %measurement interval
if info;disp(['Interval: ',int2str(param.interval)]);end
param.interval_units=fread(fid,1,'uint16'); %interval units
if info;disp(['Interval_units: ',int2str(param.interval_units)]);end
switch param.interval_units
case 0;units=' fs';
case 1;units=' ps';
case 2;units=' ns';
case 3;units=' us';
case 4;units=' ms';
case 5;units=' s';
case 6;units=' min';
case 6;units=' h';
otherwise;units=' with unknown unit';
end
if info;disp(['Sampling interval: ',num2str(param.interval),units]);end
param.trigger_sample=fread(fid,1,'uint32');
param.triggered=fread(fid,1,'uint16');
param.first_sample=fread(fid,1,'uint32');
param.sample_bytes=fread(fid,1,'uint32');
param.settings_bytes=fread(fid,1,'uint32');
param.start_date=fread(fid,1,'uint32'); %start date (days since start of year 0)
if info;disp(['Start date (days since 1 jan year 0) ',int2str(param.start_date)]);end
param.start_time=fread(fid,1,'uint32'); %start time (seconds since start of day)
if info;disp(['Start time (secondas since beginning of day) ',int2str(param.start_time)]);end
param.minimum_time=fread(fid,1,'int32');
param.maximum_time=fread(fid,1,'int32');
param.notes=fread(fid,antal_notes,'uchar')';
param.current_time=fread(fid,1,'int32');
param.spare=fread(fid,78,'uint8');
%read DATA
t = zeros(1,param.no_of_samples);
U = zeros(1,param.no_of_samples);
for p=1:param.no_of_samples %here the samples are read
t(p)=fread(fid,1,'uint')'; %time
U(p)=fread(fid,param.no_of_parameters,'float'); %no_of_parameters of data
end
%read PLS-file appended at end of PLW-file (see the end of this file for an explanation)
param.PLS=char(fread(fid,inf,'uchar'))'; %here the PLS-file is read (the first part contains strange text)
if info;disp('The param.PLS file with infomation about the run can be retreived from a third output argument');end
fclose(fid); %close file
end
Back to top
Display posts from previous:   
Post new topic   Reply to topic    Pico Technology Forum Index -> PicoLog
All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 


Pico Technology topic RSS feed 
Powered by phpBB © 2001, 2005 phpBB Group