View previous topic :: View next topic |
Author |
Message |
Serge Professional Member
Joined: 04 Mar 2002 Posts: 1480 Location: Australia
|
Posted: Mon Jan 03, 2005 8:38 am Post subject: long lists |
|
|
hi,
just thought i'd share what i have found recently concerning long lists
by using the api LB_INITSTORAGE found in the API help/resource file, i have been able to list more than 250,000 items in a list box making up more than 5 Mb in file size
i am wanting to test for 10Mb and hope to do so soon
there has been some discussion in earlier times about list sizes - may be this post will help
serge _________________
|
|
Back to top |
|
|
Skit3000 Admin Team
Joined: 11 May 2002 Posts: 2166 Location: The Netherlands
|
|
Back to top |
|
|
Boo Valued Contributor
Joined: 31 Oct 2003 Posts: 599 Location: Gulf Breeze, Florida USA
|
Posted: Mon Jan 03, 2005 12:26 pm Post subject: |
|
|
Cool, Serge! Would it be possible for you to post an example?
Thanks,
- Boo |
|
Back to top |
|
|
jules Professional Member
Joined: 14 Sep 2001 Posts: 1043 Location: Cumbria, UK
|
Posted: Mon Jan 03, 2005 12:30 pm Post subject: |
|
|
I think the issues of speed, and also of the number of elements that can be loaded into string lists (as opposed to list boxes, which are all that this message will affect) are caused by limitations of the Delphi 2 memory manager used by the current version of VDS. _________________ The Tech Pro
www.tech-pro.net |
|
Back to top |
|
|
Serge Professional Member
Joined: 04 Mar 2002 Posts: 1480 Location: Australia
|
Posted: Mon Jan 03, 2005 2:10 pm Post subject: |
|
|
skit,
i have not tried to load lists as the programs i am working on simply requires me to generate potentially huge lists and to save them
boo,
i'll see what i can do as simply copying and pasting samples from my program will not help
jules,
you are right as my findings only apply to list boxes
serge _________________
|
|
Back to top |
|
|
Serge Professional Member
Joined: 04 Mar 2002 Posts: 1480 Location: Australia
|
Posted: Tue Jan 04, 2005 7:55 am Post subject: |
|
|
hi boo,
here you go - i wrote the code on vd 4.5 and the testing was done under windows xp
Code: | Title Long Lists
DIALOG CREATE,New Dialog,-1,0,240,461
REM *** Modified by Dialog Designer on 4/01/2005 - 14:59 ***
DIALOG ADD,LIST,LIST1,10,10,220,356
DIALOG ADD,BUTTON,BUTTON1,410,90,64,24,Process
DIALOG ADD,STATUS,STATUS1,
DIALOG ADD,PROGRESS,PROGRESS1,375,10,220,24,0,,SMOOTH
DIALOG ADD,BUTTON,BUTTON2,410,20,64,24,Save
DIALOG ADD,BUTTON,BUTTON3,410,165,64,24,Open
DIALOG SHOW
rem === the 1000000 is for the upper limit for the number of lines ===
rem === the 5000000 is for the number of bytes to be allocated to the size of the list ===
%A = @sendmsg(@win(~LIST1),$01A8,1000000,5000000)
:Evloop
wait event
goto @event()
:BUTTON1BUTTON
%%number_lines = @input(Enter number of lines to generate:)
if %%number_lines
if @not(@numeric(%%number_lines))
warn You need to enter a valid number.
goto evloop
end
else
goto evloop
end
dialog disable, button1
dialog disable, button2
dialog disable, button3
%%file_size = 0
list clear, list1
%i = 0
repeat
%%new_entry = This is line number @fadd(%i,1) of %%number_lines
list add, list1, %%new_entry
rem === you need to add 2 for the end of line carriage return that equals 2 bytes ===
%%file_size = @fadd(%%file_size,@fadd(@len(%%new_entry),2))
dialog set, progress1, @name(@fdiv(@fmul(@fadd(%i,1),100),%%number_lines))
dialog set, status1, # Entries: @fadd(%i,1)@tab()@tab()File Size: %%file_size Bytes
%i = @succ(%i)
until @equal(%i,%%number_lines)
dialog enable, button1
dialog enable, button2
dialog enable, button3
warn Generation Complete!
goto evloop
:button2BUTTON
dialog disable, button1
dialog disable, button2
dialog disable, button3
%%dummy = @datetime()
list savefile, list1, @path(%0)trial.txt
%%time = @datetime(nn:ss,@fsub(@datetime(),%%dummy))
dialog enable, button1
dialog enable, button2
dialog enable, button3
if @ok()
warn File Saved in %%time
else
warn There was a problem with saving the file.
end
goto evloop
:button3BUTTON
dialog disable, button1
dialog disable, button2
dialog disable, button3
list clear, list1
%%dummy = @datetime()
list loadfile, list1, @path(%0)trial.txt
%%time = @datetime(nn:ss,@fsub(@datetime(),%%dummy))
dialog enable, button1
dialog enable, button2
dialog enable, button3
warn File Loaded in %%time
goto evloop
:Close
exit
|
some interesting things i have found
- i have been able to generate lists up to 200,000 items of around 7.5 Mb which i have saved successfully and reloaded successfully
- the code above will show you how long in minutes: seconds it took to save or load the data
- for the 200,000 items and 7.5Mb list, it took a few seconds (on my laptop) to save but 1mn and 10 seconds to load
- BUT if i load the same 200,000 entries files the first time i run the code, it took around 37 seconds to do so - loading a large file seems a lot quicker the first time the program is run, no idea why!
- as another example, i generated a list of 100,000 items and 3.7Mb - this file took 1 second to save and 17 seconds to load - BUT the first time i run the program, it loaded in 1 second!!!
i just did a quick run under windows 98se and can't seem to get past around 1167390 bytes and 32736 items in list - is there a difference between 98 and xp that would account for the difference?
serge _________________
Last edited by Serge on Tue Jan 04, 2005 7:59 am; edited 1 time in total |
|
Back to top |
|
|
Serge Professional Member
Joined: 04 Mar 2002 Posts: 1480 Location: Australia
|
Posted: Tue Jan 04, 2005 7:57 am Post subject: |
|
|
the problem under 98 seems to be the number of lines - it seems to put a limit on it in spite of the api call i used and in spite of no such limit existing under xp
i ran the script under vds 4.5 in both cases (wind 98se and xp)
serge _________________
|
|
Back to top |
|
|
jules Professional Member
Joined: 14 Sep 2001 Posts: 1043 Location: Cumbria, UK
|
Posted: Tue Jan 04, 2005 9:53 am Post subject: |
|
|
32736 does not seem like a surprising limitation in Windows 98 which is a 16-bit OS. There are a lot of 64K limitations in it, and 64K can hold 32768 16-bit pointers, or 32736 + 32 bytes of other stuff. _________________ The Tech Pro
www.tech-pro.net |
|
Back to top |
|
|
Boo Valued Contributor
Joined: 31 Oct 2003 Posts: 599 Location: Gulf Breeze, Florida USA
|
Posted: Tue Jan 04, 2005 3:08 pm Post subject: |
|
|
Thanks for the info. Serge! |
|
Back to top |
|
|
Serge Professional Member
Joined: 04 Mar 2002 Posts: 1480 Location: Australia
|
Posted: Tue Jan 04, 2005 10:55 pm Post subject: |
|
|
jules,
that win98se is a 16 bit o/s, does that mean that the list boxes are allocated 1 unit of memory, if you know what i mean
but then, what of the use of th api call to increase the amount of memory set aside for the list box?
what would be the equivalent limit under xp?
boo,
you are welcome and it was fun to do
serge _________________
|
|
Back to top |
|
|
FreezingFire Admin Team
Joined: 23 Jun 2002 Posts: 3508
|
Posted: Wed Jan 05, 2005 2:29 am Post subject: |
|
|
@Jules: Windows 98 is a 16-bit OS?
@Serge: Nice info. _________________ FreezingFire
VDSWORLD.com
Site Admin Team |
|
Back to top |
|
|
PGWARE Web Host
Joined: 29 Dec 2001 Posts: 1563
|
Posted: Wed Jan 05, 2005 6:41 am Post subject: |
|
|
A visual listbox will be much slower to load because Windows handles all the api PAINT messages. Anytime items are added to a VISIBLE listbox the scrollbar has to be painted and handled, the items, position and selection of a listbox have be maintained as well. This takes considerable memory and will cause your program to become slower.
When using visual lists, its always best to use a listbox that is eitehr hidden or DISABLED - hidden being the better option as it doesnt ahve to deal with teh numerous on screen repaint messages and redrawing. This will speed up your app greatly.
The INITSTORAGE api is generally used to speed up the usage of large lists, it really I do not think was intended to up the max amount a list box can hold. Its used to set aside a portion of memory to handle listbox storage and manipulation. |
|
Back to top |
|
|
Serge Professional Member
Joined: 04 Mar 2002 Posts: 1480 Location: Australia
|
Posted: Wed Jan 05, 2005 8:31 pm Post subject: |
|
|
Quote: | The INITSTORAGE api is generally used to speed up the usage of large lists, it really I do not think was intended to up the max amount a list box can hold. Its used to set aside a portion of memory to handle listbox storage and manipulation. |
interesting, pg ... however, i do find that without it, i cannot create very long entries in my list boxes (but only under xp)
also, the api help file says
Quote: | Allocates memory for storing list box items. An application sends this message before adding a large number of items to a list box. |
it also says
Quote: | If you overestimate, some extra memory is allocated; |
which explains why when i set aside 5Mb, i could generate a list of a bit more than 5 Mb
sorry pg!
serge _________________
|
|
Back to top |
|
|
jules Professional Member
Joined: 14 Sep 2001 Posts: 1043 Location: Cumbria, UK
|
Posted: Thu Jan 06, 2005 9:04 am Post subject: |
|
|
Serge wrote: | jules,
that win98se is a 16 bit o/s, does that mean that the list boxes are allocated 1 unit of memory, if you know what i mean
but then, what of the use of th api call to increase the amount of memory set aside for the list box?
what would be the equivalent limit under xp?
boo,
you are welcome and it was fun to do
serge |
I don't really know what you mean, but all memory allocations to the GUI subsystem, known as "resources", come from fixed size tables, which is why you have the problem of running out of resources in Win9x. Plus, many other things are kept track of using 16-bit pointers, as I said.
Win NT family systems are totally 32-bit, so the equivalent limit would be more than 2 billion. _________________ The Tech Pro
www.tech-pro.net |
|
Back to top |
|
|
jules Professional Member
Joined: 14 Sep 2001 Posts: 1043 Location: Cumbria, UK
|
Posted: Thu Jan 06, 2005 9:12 am Post subject: |
|
|
FreezingFire wrote: | @Jules: Windows 98 is a 16-bit OS? |
Yes, 'fraid so. It supports 32-bit programs, of course, but most of the core OS was just an update of Windows 3, which was 16-bit only. That's why the best thing to do with Windows 9x is consign it to the dustbin of history where it belongs.
Remember, at the time Win 95 came out, Microsoft was already developing a 32-bit Windows - NT. It was only when it became clear that NT would need more powerful computers than were commonly used at that time, that they realised they would need to extend the life of Win 3 by a few years. So 9x is not a 32-bit OS - although Microsoft tried to pretend it was - Windows 3 with some 32-bit extensions, a new desktop, and the ability to run 32-bit software. _________________ The Tech Pro
www.tech-pro.net |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You can attach files in this forum You can download files in this forum
|
|