desperate. potential places.sqlite corruption

Topic

New Reply

[PKCano]

Hello

I’ve received support from a friend already, but he couldn’t make it work. I will list what has been done:

we’ve rolled back my firefox version from 99 to 92 (sept 2021).
I have a backup of my places.sqlite from feb 2021.
We’ve tried to repair the database according to the guide, search for “Because Firefox must be closed” https://superuser.com/questions/111998/ … e-database
but there is no ERROR appearing in the sqlite dump.
I can’t even export the bookmarks. The file never shows up.
My latest places.sqlite is 320mb big and my backup 52mb.

My problem is I can’t copy/paste within that library. And that places.sqlite from feb 2021 worked flawlessly up until recently. I think the latest update 99 screwed something up.
But I don’t understand why 92 has the same problem.

I’d be grateful if someone could teamview into my computer and try something, I really want to keep using firefox.

I found your bookbackreader via https://old.reddit.com/r/firefox/commen … ll_not_be/.
Is that still working or is it deprecated?It loads, but when I click repair it crashes. https://images2.imgbox.com/98/5a/fzamWNDC_o.png
The jsonlz4 is 70 MB big.

• This topic was modified 3 years, 1 month ago by PKCano.

Reply | Quote

Replies

Hi sportman13:

What is your operating system (if Win 10 or 11 include the version and build shown at Settings | System | About | Windows Specifications) and are you still using Firefox 92 (rel. 07-Sep-2021) or have you updated back to the current Firefox v99.0.1 ?  Could you explain what problem you’re having with your Firefox browser and why you think that your places.sqlite file is corrupted?

For example, are your Firefox bookmarks missing or corrupted, and if so have you tried restoring from a recent backup as described in the Mozilla support article Restore Bookmarks From Backup or Move Them to Another Computer? If I go to Bookmarks | Manage Bookmarks | Import and Backup | Restore I’m presented with a list of bookmark backups that have been created automatically every few days in my Firefox user profile going back about five or six weeks (note that I currently have 15 bookmark backups dating back to 03-Mar-2022). I believe these are actually the .jsonlz4 files stored in my Firefox user profile in the hidden folder C:\Users\<myusername>\AppData\Roaming\Mozilla\Firefox\Profiles\<firefoxuserprofilename>\bookmarkbackups.

If that doesn’t work see the Mozilla support article Recovering Important Data from an Old Profile for instructions on how to copy both the places.sqlite and favicons.sqlite files to a new Firefox user profile to recover your bookmarks. That might not help if your current bookmark file is corrupted but it’s something to keep in mind if you run into problems using other methods.
———–
Dell Inspiron 15 5584 * 64-bit Win 10 Pro v21H2 build 19044.1586 * Firefox v99.0.1 * Microsoft Defender v4.18.2203.5-1.1.19100.5 * Malwarebytes Premium v4.5.7.186-1.0.1645

Reply | Quote

And why don’t you have a backup you can restore from?

cheers, Paul

Reply | Quote

Does ‘Export Bookmarks to HTML…’ not work either? If it does then you could save them that way and clean install Firefox (making sure to delete all traces first).

If exporting to HTML doesn’t work, then go to about:support and, under ‘Places Database’, click the button the Verify Integrity. What does it say about the places.sqlite database and the favicons.sqlite database?

Reply | Quote

restoring the lz4 files doesn’t work because it hangs at the first folder.

exporting to bookmarks.html doesn’t work either; the file never shows up.

For example, are your Firefox bookmarks missing or corrupted,

I cannot edit: can’t copy/paste/delete. Very rarely it works for about 10 seconds when I start up, but after I edit it with sqlite3 and did some PRAGMA stuff that I found on the web. I can’t make it repeat often, because I don’t know sql.

I guess my question is: is it possible to remove duplicates from within SQL?

The file is 330 MB big and has a lot of duplicates.

@sky

‘Places Database’, click the button the Verify Integrity. What does it say

I did. It didn’t help.

@Paul T

And why don’t you have a backup you can restore from?

It seems to not want to restore. Same goes for my file I thought was from 2021 but turns out is from jan 2022, which is only 50 MB big.

I’m reachable through teamviewer in case someone needs to make himself a better picture.

Reply | Quote

sportman13 wrote:

I guess my question is: is it possible to remove duplicates from within SQL?

Yes, but that may not be the issue at hand.

Something in your FF installation is broken and a restore would be the usual fix.
To do this you need to make sure you have closed all instances of FF, then restore. If that doesn’t work then something else is causing the problem.

Why not make a fresh backup of the FF installation directories, uninstall FF, then try to restore the files. If restore works, then reinstall FF and see if it has kept the settings/bookmarks. If not, close FF and restore again.

cheers, Paul

Reply | Quote

I did. It didn’t help.

Going to about:support and, under ‘Places Database’, clicking the button the Verify Integrity doesn’t fix anything, this just confirms the integrity of the places.sqlite database and the favicons.sqlite database, so, as requested, could you please tell us what it says about them. If you don’t do this then it can’t be confirmed that there is anything wrong with them.

Reply | Quote

I have to many bookmarks need to get rid of dupes is their a program to get rid dupes for Firefox ?

Reply | Quote

Not that I know of.
Open your bookmarks and edit manually, or export, tidy, delete everything, import the tidied version.

cheers, Paul

Reply | Quote

Hi sportman13:

Generally speaking, you should only use Bookmarks | Manage Bookmarks | Import and Backup | Export / Import to use an .HTML fine if you want to export / import your bookmarks to or from a different browser like IE or Chrome. If you are moving bookmarks between Firefox user profiles (even if they are on different computers) then you should back up the bookmarks in your current profile at Bookmarks | Manage Bookmarks | Import and Backup | Backup to save the .JSON file to a removable USB thumb drive or some other location and then restore them to your new profile using Bookmarks | Manage Bookmarks | Import and Backup | Restore | Choose File. If I recall correctly if you use a .HTML file with the Export / Import method you will loose the favicon (graphic image) associated with each bookmark.

The Mozilla support article Troubleshoot and Diagnose Firefox Problems might have some helpful hints. Note that a browser refresh (Step # 5 – Help | Troubleshooting Information | Refresh button) will create a new Firefox user profile (i.e., remove your add-ons and reset your custom browser settings back to their defaults) but it will retain your bookmarks, cookies, and browsing history – see Refresh Firefox – Reset Add-ons and Settings for more information. Before trying a browser refresh go to Bookmarks | Manage Bookmarks | Import and Backup | Backup and see if you can save a copy of your bookmarks as a .JSON file on a removable thumb drive or some other location just in case something goes wrong during the refresh. Out of habit, I manually backup my Firefox bookmarks to a removable thumb drive about once a week just in case the automatic backups created by Firefox are ever corrupted. Note that my last .JSON backup was about 2 MB in size and has almost 8,000 bookmarks so I don’t understand how your bookmark file could have bloated to over 70 MB.

If you reach Step # 6 of that troubleshooting guide (Create a new Firefox profile – I normally use the Firefox Profile Manager for this task) it goes one step further and creates a clean profile without retaining any personal settings like bookmarks, etc.  Before creating your new profile you should backup your bookmarks (Bookmarks | Manage Bookmarks | Import and Backup | Backup) and then restore them back into your new Firefox profile after it’s created, but if you can’t do that then you can still recover your bookmarks from your old profile.  Note that your old profile won’t be deleted from the hidden folder at C:\Users\<yourusername>\AppData\Roaming\Mozilla\Firefox\Profiles and you can always use the Firefox Profile Manager to switch back to using your old profile if the new profile doesn’t solve your problem.
————-
Dell Inspiron 15 5584 * 64-bit Win 10 Pro v21H2 build 19044.1645 * Firefox v99.0.1 * Microsoft Defender v4.18.2203.5-1.1.19100.5 * Malwarebytes Premium v4.5.7.186-1.0.1645

Reply | Quote

I have to many bookmarks need to get rid of dupes  manully takes forever I have dupes Firefox add on running for two days says searching  has not started to work any other programs which will work on Firefox ?

Reply | Quote

lmacri wrote:

Note that my last .JSON backup was about 2 MB in size and has almost 8,000 bookmarks

What search engine do you use for your bookmarks?

cheers, Paul

Reply | Quote

Hi Paul T:

Perhaps I should clarify. My image in post # 2439281 shows the size and number of items in each of my recent bookmark backups (.jsonlz4 files) created automatically by Firefox and stored in my Firefox user profile in the hidden folder at C:\Users\<myusername>\AppData\Roaming\Mozilla\Firefox\Profiles\<firefoxuserprofilename>\bookmarkbackups. I don’t know if the “number of items” shown in that image for each .jsonlz4 file directly translates to the number of individual bookmarks inside the file.

Those .jsonlz4 files created automatically by Firefox are under 1 MB in size, while the .json files I create manually (Bookmarks | Manage Bookmarks | Import and Backup | Backup) and save to a removable USB thumb drive are about 2 MB in size.

The places.sqlite file stored in my default Firefox user profile in the hidden folder at C:\Users\<myusername>\AppData\Roaming\Mozilla\Firefox\Profiles\<firefoxuserprofilename> is currently 15.4 MB in size.  The places.sqlite file holds bookmarks, the list of recent downloads and browsing history of visited sites (which I don’t clean at every browser exit) so that likely explains the larger size of my current places.sqlite file.
————
Dell Inspiron 15 5584 * 64-bit Win 10 Pro v21H2 build 19044.1645 * Firefox v99.0.1 * Microsoft Defender v4.18.2203.5-1.1.19100.5 * Malwarebytes Premium v4.5.7.186-1.0.1645

Reply | Quote

anonymous wrote:

I have to many bookmarks need to get rid of dupes is their a program to get rid dupes for Firefox ?

I used a program that is no longer available but found :
https://alternativeto.net/software/bookmark-duplicate-cleaner/

Be aware restoring from the html file will add a whole directory of duplicates. Maybe deleting all bms and then using the html file MIGHT work. Have backups available.
Also ff profiles can be very version sensitive try using the same ver FF that the profile came from and upgrading. You can experiment on a portable FF or just install another copy.

Just because you don't know where you are going doesn't mean any road will get you there.

Reply | Quote

I’ll just finish this thread. After he asked me for help and I failed initially, I told him to ask someone else, with SQL knowledge, to remove the duplicates. The person he hired simply imported bookmarks into chrome and used a duplication extension. Which worked on my end too on Brave. I didn’t try the original 332 MB file just the one I somewhat already trimmed.

Over at askwoody someone told him to verify the database via about:support. I did but I didn’t know there’s a summary that appears at the end. The next day I somewhat aimlessly tried it on my profile and realized there was a summary. Then I tried it on his places.sqlite, I had no response for a while but I let it run. After I think two or three minutes it was processed.

Since I had both his places.sqlite files and curiosity got the better of me, I was wondering where the cutoff was when Firefox would be able to read its database again and copy/paste operations would be possible again.

His moz_bookmarks alone was 1.8 million lines long. Just a bunch of duplicate nested folders within nested folders etc. He probably kept importing some .html file(s).
I extracted moz_bookmarks from places.sqlite with sqlite3 and used notepad++ and libreoffice calc to sort the most repeated folders and after I trimmed the lines to 1.4m, Firefox was able to handle it. Was a bit tricky because sometimes I cut out one folder that was a parent of children that were lost somewhere at the bottom, and the file got corrupted.
After I got it working within Firefox, it was possible that 55 folders, with no real content or bookmarks, maybe like 50-100 bookmarks across all 55 folders, took like 4-10 min to delete.
While other folders, like 200 with maybe 20 bookmarks across the 200 folders, took a few seconds. At the beginning I also had folder groups that took like 10-40 minutes to delete.

Anyway tl;dr: Firefox still has better library handling than chrome in my opinion. I loathe chrome’s and derivatives’ bookmarks/history. I don’t like the spacing; I don’t like anything about it.

 

Reply | Quote

Hello–I just happened on this site looking for why places.sqlite becomes corrupted. I haven’t yet found an answer, but I thought I’d share a simple solution. When this happens, in the FF profile I have places.sqlite.corrupted with a file size of 30,720 KB (the old file) and a new places.sqlite with a file size of 5,120 KB right below it. All I do is delete the new places.sqlite and re-name the “corrupted” file back to places.sqlite with its original file size of 30,720 KB. This gets all my history back up to the point where FF changed the file to “corrupt”, usually when I re-started it. This has worked with multiple iterations of the program (I am on FF 104.0 now, with W10 64-bit).

I frequently copy the good “places” file in the profile (<CTRL> C and immediately <CTRL> V) to get “places – copy.sqlite”. That way if re-naming the corrupted file doesn’t work I just re-name the copy file to places.sqlite and get my history back to where the copy was made. If you do this daily or every couple of days you won’t lose much when FF corrupts the file again, still for reasons unknown.

1 user thanked author for this post.

SueW

Reply | Quote

Welcome, John, and many thanks for your helpful solution! I don’t remember how I finally resolved a corrupted places.sqlite, but I’ll be noting your post as I’m sure others will as well.

Win 7 SP1 Home Premium 64-bit; Office 2010; Group B (SaS); Former 'Tech Weenie'

Reply | Quote