Upload of EPG data from XMLTV to 3PG

[JiriH]

I have tried to attach conf file to this forum, but it did not apparently work. The file, which I have copy/pasted is really event file generated using TVxb.

Part of my conf file looks like:

channel arydigital.tv
channel aljazeera.net
channel animalplanet.discoveryeurope.com
channel tsod.plus-1.animalplanet.discoveryeurope.com
channel attheraces.co.uk
channel b4utv.com
channel music.b4utv.com
channel knowledge.bbc.co.uk
channel hd.bbc.co.uk
channel news-24.bbc.co.uk
channel bbc1.bbc.co.uk
channel east.bbc1.bbc.co.uk
channel london.bbc1.bbc.co.uk
channel midlands.bbc1.bbc.co.uk
channel north.bbc1.bbc.co.uk
channel north-east.bbc1.bbc.co.uk

[mvordeme]

This does indeed look different from the TV Today configuration. I will need to look into this and improve the code. For the time being, you should comment out the channelsPath setting (add a # at the beginning of the line). Loading the configuration will take a bit longer this way, but it should work.

Regards,
-- mvordeme

[JiriH]

I have commented conf file in xmltv.properties and I was able to generate .tgd file, which are the same with .tgd files generated directly from radiotimes. Thank you for support.

So I proceeded to the last step - trying to generate .tgd file from event file, which I have published few days ago. I am receiving the message, that the format in line 1 is not correct. I have tried to modify the file to have the same header as radiotimes.xml file generated using xmltv, but still I am receiving this message. Can you help me in identification what is wrong? Should I modify dtd file?

Thanks once more for the help (I believe, that I am coming to the successful end of the story)

[JiriH]

The error message is ... at line 1: Content is not allowed in prolog

[mvordeme]

The XML prolog is this: <?xml version="1.0" encoding="utf-8" ?> and it looks fine to me. I will have a look at TVxb...

[mvordeme]

The error you reported occurs when there are characters outside the angle brackets < > in line one. Are you certain that you are reading the right file? Have you tried tgdbuilder -s? That will give you a file selection dialog for picking the files to import.

Regards,
-- mvordeme

[JiriH]

I am getting the error message in older version of TGD Builder, once I commented channel config in xmltv.properties

I have tested it in TGDBuilder 5.0. I have named source as TVxb using xmltv.But I am getting null<TVxb> once putting in channel config link to event file or nothing once commenting this line.

Can you provide me an example of channel config file (TVtoday, ..), which is recognized by TG Builder, so I could create it manually?

[mvordeme]

The error message comes from the Sun XML parser, so you will get it with any version of TGD Builder. My conf file looks like this:

Code: Alles auswählen

channel 13TH #13th Street
channel 3SAT #3SAT
channel APLAN #Animal Planet
channel ARD #ARD
channel ARENA #Arena
channel ARTE #ARTE
channel ATV #ATVplus
channel AXN #AXN
channel BBC #BBC WORLD
channel BBC-P #BBC Prime
channel BIBEL #Bibel TV
...

Your file lacks the description behind the #, so there is probably not a lot I can do with files like that. I have to think about it.

Even if you manage to have the conf file scanned by TGD Builder, the parser error will still occur when reading the events, so you should check whether your eventsPath points at the right file. I suspect that it is not pointing at an XML file.

Regards,
-- mvordeme

[JiriH]

I have cretaed .conf file matching your template and I got correctly the list of channels in TGD Builder 5.0. When I tried to generate the .tgd file I got the same error message as in older version.

I am quite sure, that the file selected is correct ( I have replaced and renamed the file with xmltv file generated by xmltv for radiotimes, renamed one channel to CT1 matchin TVxb channel and I did not get the error message. I am also confused, because when I replace the beinning of the Tvxb file with radiotimes file, still I am getting the error message.

I am trying to attch the event file to this message, but I can not make it

[JiriH]

Hallo,

I will send you the file immediately.

During the evening I have investigated the problem in detail, modifying gradually file from xmltv radiotimes (working properly with TGD Buikder) and I have found, that the problem is in the encoding of the file. xmltv radiotimes file is using ANSI encoding and TVxb file is using utf-8. It is sufficient to open xmltv radiotimes file in notepad and save it as ANSI and TGD Buiilder processeses it properly. My suspicion is, that the problem is in control characters <CR> <LF>.

Resolving this I was able to generate .TGD files, finding another problem (hope the last one):

While event file contains proper Slavic charters:

<programme start="20080515083000 +0200" stop="20080515090000 +0200" channel="ct1">
<title lang="cs">Kouzelná školka</title>
<desc>Pohádky a povídání pro nejmenší k podívání</desc>
</programme>

The generated .TGD file distorts them:

1.CT1 2008-05-15 08:30 30 Kouzelná ?kolka Pohádky a povídání pro nejmen?í k podívání

3PG definitely supports these characters, because I can see them after the scan.

Thanks a lot for help and support

[mvordeme]

Hi Jiri,

the problem with the XML prolog is caused by three "invisible" characters at the beginning of the first line. My email client (Thunderbird) shows them as . Currently, you will need to remove them with a suitable text editor. I was able to do so using WinVi. UltraEdit shows them as well if you switch to hex mode. It should be no problem to ignore leading content in the next version of TGD Builder.

The problem with the Czech characters is one of the proper encoding. I am going to look into this and see what can be done. Maybe the encoding of the XML file can be carried over.

Regards,
-- mvordeme

P.S./b] How did you get the symbols into 3PG? Did you modify the TGD manually?

[mvordeme]

I ran a few tests, today. I managed to modify the way TGD Builder is reading the XML data, which miraculously solved the problem with the invisible content at the top of the file. Unfortunately, the ISO-8859-1 encoding does not contain all Slavic characters, so I changed the output to ISO-8859-15, which is supposed to contain at least a few of them, but they did not come up right in 3PG. If I switch to UTC-8, I get multibyte-characters for almost everything which apparently are not recognized by 3PG.

I can try to get hold of cactux and ask him whether this is possible at all and which encoding to use. If there is no solution for the problem, I could at least try to replace the characters with non-accented ones, so you would get an idea of the original text.

Please let me know whether you need a fix for the invisible content bug, soon.

Regards,
-- mvordeme

P.S./b] I've now also tried the Windows-1252 encoding, but 3PG definitely only reads ISO-8859-1 properly.

[mvordeme]

The current version fixes the problem with the invisible characters at the top of the file. If I find the time, I will map unmapped multi-byte characters to their closest equivalent (e.g. š to s) in the next version.

Regards,
-- mvordeme

[mvordeme]

I have built a character converter into the next version of TGD Builder. Since it appears that 3PG can only import ISO-encoded data, the converter tries to match the rest of the Latin Unicode characters as closely as possible. Here is a sample entry from an old HRT programme quote="xmltv"]Ponos Ratkajevih, TV serija (kod. na sat.)

Josipu na sastanku predla?u novu funkciju. Veronika tje?i uplakanu Terezinu majku. Lotte i Bela raspravljaju o ljubavi. Slavko u Londonu vodi Nemanju u kupleraj, gdje Nemanja upozna Valentinu. Hana prepričava Terezi sve ?to je čula o Lisi od svog oca. Helena se budi u bolnici, pokraj nje su Krsto i Bijelić. Hebb jasno daje do znanja Beli da ga smeta ?to je Nemanja Srbin.

Bo?idar Alić (Karlo pl. Ratkaj), Sanja Vejnović (Veronika pl. Ratkaj), Janko Popović-Volarić (Krsto pl. Ratkaj), Marijana Mikulić (Izabela pl. Ratkaj), Vanessa Radman (Lisa Cohen), Ljubica Jović (grofica Antonija Ratkaj), Zijad Gračić (Josip Juri&#263 [/quote]This is what TGD Builder used to turn it into quote="TGD Builder 0.5"]Ponos Ratkajevih, TV serija (kod. na sat.)

Josipu na sastanku predla?u novu funkciju. Veronika tje?i uplakanu Terezinu majku. Lotte i Bela raspravljaju o ljubavi. Slavko u Londonu vodi Nemanju u kupleraj, gdje Nemanja upozna Valentinu. Hana prepri?ava Terezi sve ?to je ?ula o Lisi od svog oca. Helena se budi u bolnici, pokraj nje su Krsto i Bijeli?. Hebb jasno daje do znanja Beli da ga smeta ?to je Nemanja Srbin.

Bo?idar Ali? (Karlo pl. Ratkaj), Sanja Vejnovi? (Veronika pl. Ratkaj), Janko Popovi?-Volari? (Krsto pl. Ratkaj), Marijana Mikuli? (Izabela pl. Ratkaj), Vanessa Radman (Lisa Cohen), Ljubica Jovi? (grofica Antonija Ratkaj), Zijad Gra?i? (Josip Juri?)[/quote]This is the same text with the new character conversion quote="TGD Builder 0.6"]Ponos Ratkajevih, TV serija (kod. na sat.)

Josipu na sastanku predlazu novu funkciju. Veronika tjesi uplakanu Terezinu majku. Lotte i Bela raspravljaju o ljubavi. Slavko u Londonu vodi Nemanju u kupleraj, gdje Nemanja upozna Valentinu. Hana prepricava Terezi sve sto je cula o Lisi od svog oca. Helena se budi u bolnici, pokraj nje su Krsto i Bijelic. Hebb jasno daje do znanja Beli da ga smeta sto je Nemanja Srbin.

Bozidar Alic (Karlo pl. Ratkaj), Sanja Vejnovic (Veronika pl. Ratkaj), Janko Popovic-Volaric (Krsto pl. Ratkaj), Marijana Mikulic (Izabela pl. Ratkaj), Vanessa Radman (Lisa Cohen), Ljubica Jovic (grofica Antonija Ratkaj), Zijad Gracic (Josip Juric)[/quote]It probably still looks funny, but a lot better than before. If anyone sees room for improvements, I am open for suggestions.

Regards,
-- mvordeme