Vanishing Padding

[keith_leitch]

Every so often, my timers stop picking up the padding information and suffix settings from SmartEPG searches. That is, recordings will start recording with no padding at the start and finish, even though my searches are set to 10 minutes before and 20 minutes after. Also, they will start suffixing multiple episodes with -2 -3, etc., even though my settings are for date and time.

In the past, I have been able to fix this just by modifying one search and re-saving. This works, but does not correct existing timers. Regardless, we often record well in advance, so it could take awhile to notice this has happened.

I thought I had read that this was a bug that had been fixed in a recent version. I am running v7.2b, which TapToDate believes is the latest. Can anyone advise?

[FireBird]

At least, I would need the "SmartEPG_TMS.log" and "FileDB.log" logs and a hint, which recording went wrong.

[keith_leitch]

OK, I will see what I can do. I may have already deleted all of these recordings, but I will check; if not, I will save them for next time. Are you saying that you believe this bug should be fixed?

[Twilight]

you still have the logs. so just send firebird the logs with the name of the recordings...

twilight

[keith_leitch]

I understand, but I do not remember which recordings were effected. I have to see whether I still have them on my drives, so that I can tell Alex the names.

[keith_leitch]

OK, Got it:

Two affected recordings were:

'Major Crimes - New' (2016-11-20 21:00

'Fresh Off The Boat' (2016-11-21 20:00

The second one should show the -2 suffix, rather than the date/time suffix that I have in the settings. Both had the pre- and post-padding cut off.

Thanks for having a look.

[FireBird]

I’m not sure what the exact reason was but I’ve found some interessting entries in the log. From time to time, you have messages like the following one:

Code: Alles auswählen

TimerDB: Flash timer … has an ID but no corresponding DB timer

That means that SmartEPG is not aware about a timer held in the Toppy’s flash memory and synchronises its database. If you don’t have another TAP that creates timers while SE is not active, this would point to a damaged SE timer database (Timer.db). According to the log, the last one was on 2016-11-17 22:42. There were several other similar events but that one was the largest.

In addition, SE logged 68 unexpected reboots during the last month. Either you should fix an unstable machine or you should move your script development to a development Toppy because these crashes may damage the file system and result in unexpected behaviour of the TAPs.

[keith_leitch]

Thanks.

I do run jdaEPGMoreInfo, and occasionally use it to create a timer. However, SE is always active when I do so.

I suspect the first problem could be caused by unexpected changes in the EPG, which are frequent here in Australia. Does that make sense to you? I am assuming that the "trick" of resetting and saving one timer replaces Timer.db. If not, should i be fixing it? Perhaps I should manually delete it and let the software recreate it?

I appreciate your time looking at this.

EDIT: I was thinking about this on my bike, and wonder how a damaged Timer.db would cause this problem. Wouldn't SE still pick up the settings to create any new timers? The problem is that it is not, unless I change a timer and redo the search.

However, the date you noted does seem to correspond to the latest problem.

I also realised that I haven't seen a bad reboot for a long time, now. Are you talking about the lines that say "Reboot Detected?" If so, this is probably when I use my WiFi power switch to start the toppy from a remote location. I have it scripted to ping it first, so it only happens if the machine is actually in standby. Before they went under in Australia, I checked this with Toppro, who confirmed that this will not effect the file system.

[FireBird]

I suspect the first problem could be caused by unexpected changes in the EPG, which are frequent here in Australia.

SE should be able to cope with a lot of different types of unexpected changes, like changing start times or durations, or even the worst case of changing the event id of the same programme.

I am assuming that the "trick" of resetting and saving one timer replaces Timer.db.

It just adds and deletes one record of the timer database.

If not, should i be fixing it? Perhaps I should manually delete it and let the software recreate it?

You have to delete both the timer.db and the timers in the flash memory, otherwise all flash timers will be added as manual timers and the next EPG search might create search timers for the same time period. There is some logic that might detect these dupe timers for the same programme and combine them to the correct search timer.

But this procedure normally does not need to be done. The question is why your db gets damaged from time to time.

Wouldn't SE still pick up the settings to create any new timers? The problem is that it is not, unless I change a timer and redo the search.

The flash has much less information then what is kept in the timer.db and if it gets damaged and the flash timers are imported into the db, a lot of information is lost. But see above.

I also realised that I haven't seen a bad reboot for a long time, now. Are you talking about the lines that say "Reboot Detected?" If so, this is probably when I use my WiFi power switch to start the toppy from a remote location.

Yes, these are the lines. In the case of a regular shutdown, a ?SE shutdown finished? should be logged just before the power goes off and in case of a ?Reboot detected?, that didn?t happen. Therefore it seems that the power gets pulled before the Toppy had a chance to shut down correctly.

[Twilight]

se dedects a reboot when it is not stopped normaly...so if you are right and the trick with the ping works this is not the problem ...

twilight

[keith_leitch]

[quote="FireBird"]
It just adds and deletes one record of the timer database.

[/quote]

...but it must do more than this. When I have the problem, all timers are created incorrectly until I make one small change to one (only one) and launch the search again. From then on, the timers create correctly until the next time there is a problem.

FYI, the change I normally make is to slightly change the window of search times: e.g. instead of searching until 00:00, I search until 23:59. This doesn't matter, though; just about any change will do. Once that is done, new timers then start to appear again with padding and with the correct suffix. This lasts a long time.

I thought it was you who advised me to use this fix in the past: somebody did.

[quote="FireBird"]
But this procedure normally does not need to be done. The question is why your db gets damaged from time to time.
[/quote]

I agree, but also wonder whether the db is damaged and has stayed damaged. I will do this as a clean-up, as long as you don't think it will do any harm.


[quote="FireBird"]
Therefore it seems that the power gets pulled before the Toppy had a chance to shut down correctly.[/quote]

OK; it is not because of the wifi switch then. I have tested it many times while I am watching, and the script works reliably. (Thanks, Twilight, for confirming this).

[FireBird]

I will do this as a clean-up, as long as you don't think it will do any harm.

There's no harm except you'll lose manually created timers. Search timers will be recreated without problems.

OK; it is not because of the wifi switch then. I have tested it many times while I am watching, and the script works reliably. (Thanks, Twilight, for confirming this).

The log is clear in that point and if you can find the reason and fix it, I’m sure that most of your problems will be gone.

[keith_leitch]

Thanks again. I will keep an eye out.

FYI, I found this old reply from you here:

https://topfield.forumchitchat.com/post ... rum=375589

Even though the search editor will show the expected naming option, it doesn’t get written into the Search.txt until you save an arbitrary search and therefore creates name-only timers. If you have changed back to 7.1, please open any search and save immediately (green). No need to change anything. This should fix all searches.

This is when I first started changing just one search whenever I notice this problem. You seemed to be saying that saving one search would correct the naming for everything, all at once, in the Search.txt file; that has always seemed to work. Could it be this file that is corrupt and causing me trouble? Will it do any harm to delete it, too, when I clear the others?

[FireBird]

That was a solution for a very specific problem and has nothing to do with the current problem. Please fix your shutdown issue and then we’ll see if further troubleshooting is necessary.

[keith_leitch]

I apologise if I bothered you. I was just letting you know that the same solution works to repair my timers each time this happens. I thought it might interest you to know this.

I appreciate everything you've contributed to Topfield owners, and didn't mean to imply otherwise.

As far as I can tell, my system is not rebooting unexpectedly anymore. I think it was related to Beta-testing a TAP for a developer, which I have now removed. I will keep an eye on the SE logs, though, now that I know what to look for. Thank you for pointing it out to me.

[FireBird]

SE detects that it has been launched though it has never seen the shutdown. Even if it calls it reboot in the log, it is clear that this is not a reboot as we know it. Wait some days and recheck the log to see if there are still "reboot" entries. If yes, are there "SE shutdown finished" messages? If not, your Toppy shuts down before the TAPs are able to stop.

You don’t bother anybody and you’re invited to ask whenever you like. But we need to investigate from top down. It doesn’t make sense to change config files if the log points to shutdown problem. And this is the point where you have to try to find out what’s going.

Cheers,
Alex

[keith_leitch]

I've had two crashes since, both due to power outages during storms. The problem with the padding has not recurred, but I don't think it would every time.

I wonder if the crashes could be causing another problem I have had for a long time:

After awhile, the first-press SE Infobox will no longer let me scroll through it. The only way I have been able to fix this is to first roll back SE to a version before 7.0 (I think I just use 6.2), then update to 7.0, and then to 7.2.

If I skip any of these steps, the Infobox stays frozen. If I follow them, it works again, but then stops working a few weeks or months later.

Can you think of any files that would be restored or replaced by following those steps?

[FireBird]

The problem with the padding has not recurred, but I don't think it would every time.

The main question is if the log now suggests that your Toppy is shutting down cleanly, i.e. the log contains the “SE shutdown finished” entry for every shutdown.

After awhile, the first-press SE Infobox will no longer let me scroll through it.

Currently I have no idea. If it happens again, please create a backup and send it to me and I’ll try if I can reproduce it.

[keith_leitch]

Yes, it is shutting down cleanly, except for the twice that I had a power failure.

As for the infobox: it is currently misbehaving, but it has been for a long time (maybe months). I just haven't bothered to fix it. Is that any good to you, or should I fix it and send the minute it happens again?

[FireBird]

Just send a backup and I'll see if I can reproduce the problem.