Rocksolid Light

Welcome to Rocksolid Light

mail  files  register  newsreader  groups  login

Message-ID:  

For large values of one, one equals two, for small values of two.

computers / Rocksolid Nodes Announce / Branch status for RSLight

SubjectAuthor
* Branch status for RSLightRetro Guy
`* Branch status for RSLightRetro Guy
 `* Branch status for RSLightvga256
  `* Branch status for RSLightRetro Guy
   `* Branch status for RSLightvga256
    `- Branch status for RSLightRetro Guy

1
Branch status for RSLight

<7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org>

  copy mid

https://news.novabbs.org/computers/article-flat.php?id=444&group=rocksolid.nodes.announce#444

  copy link   Newsgroups: rocksolid.nodes.announce
Date: Mon, 14 Aug 2023 13:34:06 +0000
Subject: Branch status for RSLight
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
From: retro.guy@rocksolidbbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.announce
X-Rslight-Site: $2y$10$2qZFDpfuQ0lVYMPyA5QOs.Tp8HYbx2Nt0P5HHZZTbGudwtIQZrkbu
X-Rslight-Posting-User: 91053d4a47d51b416144568e5a1040f05e31ed1b
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
User-Agent: Rocksolid Light 0.9.0
Organization: Rocksolid Light
Message-ID: <7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org>
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf
?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B
H0=hZt'[%)4kG|
 by: Retro Guy - Mon, 14 Aug 2023 13:34 UTC

The 'messaging' branch has been merged into master.

There is a new branch, 'overview'. You don't want to pull from this branch atm as it contains major changes and will not work with the overview database previous to these changes.

Currently, https://news.novabbs.org is running with all these changes, and seems to be running well and substantially faster.

The changes are as follows:

Completely remove the use of the per group flat file overview databases, in favor of only the .db3 site overview. The .db3 was already used, so making it match the features of the flat file db made sense, plus, the .db3 is much faster.

Modified import-db3.php to bring 'articles-overview.db3' up to date (to work with this version). It requires a one-time run of import-db3.php to update the .db3, then you can update to this version (0.9.0).

Renamed the function rslight_db_open to overview_db_open throughout the code. It makes much more sense, and easier to avoid using the wrong function.

During the process of making these changes, several fixes for tradspool were made.

There is a release link on github.com for version 0.8.5, which contains all the code previous to these changes.

--
Retro Guy

Re: Branch status for RSLight

<60a5a58fdaa965ca1d88bd7f72fa88d1@news.novabbs.org>

  copy mid

https://news.novabbs.org/computers/article-flat.php?id=447&group=rocksolid.nodes.announce#447

  copy link   Newsgroups: rocksolid.nodes.announce
Date: Tue, 15 Aug 2023 13:57:21 +0000
Subject: Re: Branch status for RSLight
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
From: retro.guy@rocksolidbbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.announce
X-Rslight-Site: $2y$10$NRIw9wKDfnIb1pPFMM6E3eHD2jUdkLeO0x7ioRIvKmDE/y362aDv6
X-Rslight-Posting-User: 91053d4a47d51b416144568e5a1040f05e31ed1b
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
User-Agent: Rocksolid Light 0.9.0
References: <7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org>
Organization: Rocksolid Light
Message-ID: <60a5a58fdaa965ca1d88bd7f72fa88d1@news.novabbs.org>
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf
?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B
H0=hZt'[%)4kG|
 by: Retro Guy - Tue, 15 Aug 2023 13:57 UTC

Retro Guy wrote:

> Completely remove the use of the per group flat file overview databases, in favor of only the .db3 site overview. The .db3 was already used, so making it match the features of the flat file db made sense, plus, the .db3 is much faster.

This change seems substanially faster since we're not scrolling through a huge flat file over and over again. Just reading/writing from a sorted database.

--
Retro Guy

Re: Branch status for RSLight

<f5586f20dcb16e14b336ce491167ab2b@news.novabbs.org>

  copy mid

https://news.novabbs.org/computers/article-flat.php?id=448&group=rocksolid.nodes.announce#448

  copy link   Newsgroups: rocksolid.nodes.announce
Date: Tue, 15 Aug 2023 13:59:24 +0000
Subject: Re: Branch status for RSLight
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
From: vga@vga256.com (vga256)
Newsgroups: rocksolid.nodes.announce
X-Rslight-Site: $2y$10$r7c8ZQa2M2RCpO75D/nAP.X1WzLhaqDmLAEOsec/Fw2Eec7TWWK0C
X-Rslight-Posting-User: 9d50238f0c8e65588ce75c7d7eaad42d47312328
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
User-Agent: Rocksolid Light 0.9.0
References: <7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org> <60a5a58fdaa965ca1d88bd7f72fa88d1@news.novabbs.org>
Organization: Rocksolid Light
Message-ID: <f5586f20dcb16e14b336ce491167ab2b@news.novabbs.org>
 by: vga256 - Tue, 15 Aug 2023 13:59 UTC

Very cool. I had no idea it was so expensive reading the flatfile.

--
Posted on Rocksolid Light

Re: Branch status for RSLight

<6d161fc5c1f4989eeba22f7de5ad4423@news.novabbs.org>

  copy mid

https://news.novabbs.org/computers/article-flat.php?id=449&group=rocksolid.nodes.announce#449

  copy link   Newsgroups: rocksolid.nodes.announce
Date: Tue, 15 Aug 2023 14:33:05 +0000
Subject: Re: Branch status for RSLight
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
From: retro.guy@rocksolidbbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.announce
X-Rslight-Site: $2y$10$utkBEcIgCL38e6v0GgacW.0c9.o4.IXzCgQqBCsMQOXehayE1coru
X-Rslight-Posting-User: 91053d4a47d51b416144568e5a1040f05e31ed1b
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
User-Agent: Rocksolid Light 0.9.0
References: <7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org> <60a5a58fdaa965ca1d88bd7f72fa88d1@news.novabbs.org> <f5586f20dcb16e14b336ce491167ab2b@news.novabbs.org>
Organization: Rocksolid Light
Message-ID: <6d161fc5c1f4989eeba22f7de5ad4423@news.novabbs.org>
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf
?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B
H0=hZt'[%)4kG|
 by: Retro Guy - Tue, 15 Aug 2023 14:33 UTC

vga256 wrote:

> Very cool. I had no idea it was so expensive reading the flatfile.

Sorting in sqlite makes a big difference. Speed is greatly improved if you create an INDEX on each column you plan to search for things. We have always had these indices in the dbs, but the flat file overview was really slowing things down.

For example, www.novabbs.com (com, not org) currently contains 3,640,739 articles. That's 3,640,739 lines in the overview file. That's a lot to search through, or delete one line from. This site is not yet running the new code.

--
Retro Guy

Re: Branch status for RSLight

<254c53e2c2e9d341868e4ebca1f2f4db@news.novabbs.org>

  copy mid

https://news.novabbs.org/computers/article-flat.php?id=450&group=rocksolid.nodes.announce#450

  copy link   Newsgroups: rocksolid.nodes.announce
Date: Wed, 16 Aug 2023 00:44:17 +0000
Subject: Re: Branch status for RSLight
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
From: vga@vga256.com (vga256)
Newsgroups: rocksolid.nodes.announce
X-Rslight-Site: $2y$10$kk7KXtJSwBYLzkJoYRENXO.KPCyoKeFaHxTYoMsFfSmihiO0.jpV6
X-Rslight-Posting-User: 9d50238f0c8e65588ce75c7d7eaad42d47312328
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
User-Agent: Rocksolid Light 0.9.0
References: <7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org> <60a5a58fdaa965ca1d88bd7f72fa88d1@news.novabbs.org> <f5586f20dcb16e14b336ce491167ab2b@news.novabbs.org> <6d161fc5c1f4989eeba22f7de5ad4423@news.novabbs.org>
Organization: Rocksolid Light
Message-ID: <254c53e2c2e9d341868e4ebca1f2f4db@news.novabbs.org>
 by: vga256 - Wed, 16 Aug 2023 00:44 UTC

Yeow! one of those scale issues I never think about.

--
Posted on Rocksolid Light

Re: Branch status for RSLight

<f55718d12903a1172bf2ecec47b49be6@news.novabbs.org>

  copy mid

https://news.novabbs.org/computers/article-flat.php?id=451&group=rocksolid.nodes.announce#451

  copy link   Newsgroups: rocksolid.nodes.announce
Date: Wed, 16 Aug 2023 16:45:32 +0000
Subject: Re: Branch status for RSLight
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
From: retro.guy@rocksolidbbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.announce
X-Rslight-Site: $2y$10$CGcBZF3Udshu1A6bsIWYi.9WBqnu0oTd.r4RgEuZ5cB7rJkspK8iC
X-Rslight-Posting-User: 91053d4a47d51b416144568e5a1040f05e31ed1b
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
User-Agent: Rocksolid Light 0.9.0
References: <7836e57bd097594c8d0c4008d3179bb9@news.novabbs.org> <60a5a58fdaa965ca1d88bd7f72fa88d1@news.novabbs.org> <f5586f20dcb16e14b336ce491167ab2b@news.novabbs.org> <6d161fc5c1f4989eeba22f7de5ad4423@news.novabbs.org> <254c53e2c2e9d341868e4ebca1f2f4db@news.novabbs.org>
Organization: Rocksolid Light
Message-ID: <f55718d12903a1172bf2ecec47b49be6@news.novabbs.org>
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf
?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B
H0=hZt'[%)4kG|
 by: Retro Guy - Wed, 16 Aug 2023 16:45 UTC

vga256 wrote:

> Yeow! one of those scale issues I never think about.

It's a big file, lol :)

Branch 'overview' now contains the changes to remove the use of overview flat file, and also the changes to add history.db3 to track (and not reuse) deleted article numbers.

Note: DO NOT UPGRADE to this branch yet, as the old articles-overview.db3 is not structured to handle the new overview. A task must be run first to upgrade the database.

I'll post here after testing a new ways that don't take forever to run. My initial test was to just write it from each group.db3, but there are faster ways that I'm looking into right now.

--
Retro Guy

1
server_pubkey.txt

rocksolid light 0.9.8
clearnet tor