Extracting and structuring data from a xml-like file in R

26 Views Asked by At

I would like to exploit information within this file via R.

It looks like a XML file, but standard XML::xmlToDataFrame() will return an error (I assume it is related to the fact that the attributes <Nationality> for instance has several values.

I am basically aiming to create a simple data frame, with attributes as column (Country, CountryCode, CountryName, ..., Nationality, Capital, etc.) and countries in rows, and collapsing when necessary multiple values (e.g. for Nationality) in a single elements using "," for instance.

A list where each element would be a country, and then each of its element would be the attributes would also be OK.

How can we achieve that in R please?

[edit] Adding file extract as requested by user Parfait (removing comments in header, and providing a single country):

<Doc>Version: CountryInfo.140728.txt</Doc>
<Country>
<CountryCode>AFG</CountryCode>
<CountryName>AFGHANISTAN</CountryName>
<COW-Alpha>AFG</COW-Alpha>
<COW-Numeric>700</COW-Numeric>
<FIPS-10>AF</FIPS-10>
<IMF>512</IMF>
<ISO3166-alpha2>AF</ISO3166-alpha2>
<ISO3166-numeric>004</ISO3166-numeric>
<ISO3166-alpha3>AFG</ISO3166-alpha3>
<Nationality>
    AFGHAN_
    AFGANISTAN_
    AFGHANESTAN_
    AFGHANSITAN_
    DOWLAT_E_ESLAMI_YE_AFGHANESTAN_
    SOVIET_OCCUPIED_AFGHANISTAN_
    AFGAHANISTAN_
    AFGHANASTAN_
    AFGANHISTAN_
    AFGHINASTAN_
    AFFGHANISTAN_
    THE_GRAVEYARD_OF_EMPIRES_
    AFGJANISTAN_
    AFEGANISTAO_
    CHAPARHAR_
</Nationality>
<Capital>
    KABUL_
</Capital>
<MajorCities>
    HERAT_
    JALALABAD_
    KANDAHAR_
    QANDAHAR_
    MAZAR-I-SHARIF_
</MajorCities>
<Leaders>
    <Kings>
        MOHAMMAD_ZAHIR_SHAH_                [19331108 - 19730717]  [B:19140101]  [D:20070723]
    </Kings>
    <Chairman of the military council>
        ABDUL_QADIR_                        [19780427 - 19780430]  [B:19440101]
    </Chairman of the military council>
    <Presidents of the revolutionary council>
        NUR_MOHAMMAD_TARAKI_                [19780430 - 19790916]  [B:19170101]  [D:19790914]
        HAFIZULLAH_AMIN_                    [19790916 - 19791227]  [B:19290101]  [D:19791227]
        BABRAK_KARMAL_                      [19791227 - 19861124]
        HAJI_MOHAMMAD_CHAMKANI_             [19861124 - 19870930]  [B:19470101]
        MOHAMMAD_NAJIBULLAH_                [19870930 - 19871130]
    </Presidents of the revolutionary council>
    <President>
        SARDAR_MOHAMMAD_DAUD_KHAN_          [19730717 - 19780427]  [B:19090101]  [D:19780428]
        MOHAMMAD_NAJIBULLAH_                [19871130 - 19920416]
        ABDUL_RAHIM_HATEF_                  [19920416 - 19920428]  [B:19260101]
        SIBGHATULLAH_MOJADEDI_              [19920428 - 19920628]  [B:19250101]
        BURHANUDDIN_RABBANI_                [19920628 - 19960927]  [B:19400101]
        BURHANUDDIN_RABBANI_                [20011113 - 20011222]
        HAMID_KARZAI_                       [>20011222]  [B:19570101]
    </President>
    <Heads of the supreme council>
        MULLAH_MOHAMMAD_RABBANI_                [19960927 - 20010416]  [B:19560101]  [D:20000421]
        MAWLAWI_ABDUL_KABIR_                    [20010416 - 20011113]
    </Heads of the supreme council>
    <Prime ministers>
        MOHAMMAD_NUR_AHMAD_ETEMADI_         [19671101 - 19710609]  [B:19210101]  [D:19790801]
        ABDUL_ZAHIR_                        [19710609 - 19721212]  [B:19100101]  [D:19820101]
        MOHAMMAD_MUSA_SHAFIQ_               [19721212 - 19730717]  [B:19320101]  [D:19790101]
        NUR_MOHAMMAD_TARAKI_                [19780501 - 19790327]
        HAFIZULLAH_AMIN_                    [19790327 - 19791227]
        BABRAK_KARMAL_                      [19791227 - 19810611] 
        SULTAN_ALI_KESHTMAND_               [19810611 - 19880526]  [B:19350101]
        MOHAMMAD_HASSAN_SHARQ_              [19880526 - 19890221]  [B:19250101]
        SULTAN_ALI_KESHTMAND_               [19890221 - 19900508]
        FAZAL_HAQ_KHALIQYAR_                    [19900508 - 19920415]  [B:19340101]  [D:20040716]
        ABDUL_SABUR_FARID_KUHESTANI_            [19920706 - 19920815]  [B:19520101]  [D:20070503]
        GULBUDDIN_HEKMATYAR_                    [19930617 - 19940628]  [B:19470101]
        ARSALA_RAHMANI_                     [19941101 - 19950101]  [B:19370101]
        AHMAD_SHAH_AHMADZAI_                    [19950101 - 19960626]  [B:19440101]
        GULBUDDIN_HEKMATYAR_                    [19960626 - 19960927]
        ABDUL_RAHIM_GHAFOORZAI_             [19970811 - 19970821]  [D:19970821]
    </Prime ministers>
</Leaders>
<Government>
    ABDULLAH_RAMIN_ [AFGGOVAGR 20050101 - 20050331] # Minister of Agriculture
    ABDULLAH_WARDAK_    [AFGGOV <20041231]  # Minister of Martyrs and Disabled
    ABDUL_ALI_  [AFGGOV <20040931]  # Minister of Public Works
    ABDUL_AZIMI_    [AFGGOVJUD 20061001 - 20070531] # Chief Justice, Supreme Court
    ABDUL_HADI_ARGHANDIWAL_ [AFGGOV >20100201]  # Minister of Economy
    ABDUL_KARIM_BARAWI_ [AFGGOV 20050101 - 20060531]    # Minister of Border and Tribal Affairs
    ABDUL_KARIM_BRAHAWI_    [AFGGOVREF 20090701 - 20100331] # Minister of Refugees and Repatriation
    ABDUL_KARIM_KHALILI_    [AFGGOV >20030101]  # Vice President
    ABDUL_KARIM_KHURAM_ [AFGGOVMED 20061001 - 20100131] # Minister of Youth and Culture
    ABDUL_QADIR_FITRAT_ [AFGGOVBUS >20071201]   # Governor, Central Bank
    ABDUL_QODUS_HAMIDI_ [AFGGOV >20100701]  # Minister of Public Works
    ABDUL_RAHIM_KARIMI_ [AFGGOVJUD <20041231]   # Minister of Justice
    ABDUL_RAHIM_WARDAK_ [AFGGOVMIL >20050101]   # Minister of Defense
    ACTING_MINISTER_ABDUL_RAHIM_    [AFGGOVREF 20100401 - 20100631] # Minister of Refugees and Repatriation "Acting"
    AHMAD_MOQBEL_ZARAR_ [AFGGOV 20051101 - 20060531][AFGGOV 20070401 - 20081031][AFGGOVCOP >20100201]   # Minister of Interior & Minister of Counternarcotics
    AHMAD_MOSHAHED_ [AFGGOVEDU 20041001 - 20041231] # Minister of Education
    AHMAD_YUSUF_NURISTANI_  [AFGGOVAGR <20041231]   # Minister of Irrigation and Environment
    AHMAD_ZIA_MASOOD_   [AFGGOV 20050201 - 20091231]    # Vice President
    AKBAR_AKBAR_    [AFGGOVREF 20060601 - 20071131] # Minister of Refugees
    ALI_AHMAD_JALALI_   [AFGGOV 20031001 - 20051031]    # Minister of Interior
    AMENA_AFZALI_   [AFGGOV 20050201 - 20060531][AFGGOVLAB >20100201]   # Minister of Youth & Minister of Martyred, Disabled, Labor, and Social Affairs
    AMINA_AFZALI_   [AFGGOV 20050201 - 20060531][AFGGOVLAB >20100201]   # Minister of Youth & Minister of Martyred, Disabled, Labor, and Social Affairs
    AMIRZAI_SANGIN_ [AFGGOVMED >20050101]   # Minister of Communications
    ANWAR_UL_HAQ_AHADY_ [AFGGOV 20050101 - 20090231][AFGGOVBUS 20031001 - 20041231][AFGGOVBUS >20100701]    # Minister of Finance & Governor, Central Bank & Minister of Commerce and Industry
    ARSALA_JAMAL_   [AFGGOV 20100401 - 20100631]    # Minister of Border and Tribal Affairs "Acting"
    ASADULLAH_KHALID_   [AFGGOV 20090101 - 20100331][AFGGOV >20100701]  # Minister of Border and Tribal Affairs
    ASHRAF_GHANI_   [AFGGOV <20041231]  # Minister of Finance
    ASIF_RAHIMI_    [AFGGOVAGR >20081101]   # Minister of Agriculture
    AZAM_DADFAR_    [AFGGOVREF 20050101 - 20060531] # Minister of Refugees
    BISMILLAH_BISMEL_   [AFGGOVBUS 20040601 - 20041231] # Minister of Civil Aviation and Tourism
    BISMULLAH_MUHAMMADI_KHAN_   [AFGGOV >20100701]  # Minister of Interior
    DAOUD_ALI_NAJAFI_   [AFGGOVBUS >20100801]   # Minister of Transportation "Acting"
    EHSAN_ZIA_  [AFGGOVAGR 20060601 - 20100131] # Minister of Rural Development
    ENAYATOLLAH_QASEMI_ [AFGGOVBUS 20050101 - 20060531] # Minister of Transport
    ENAYATULLAH_NAZERI_ [AFGGOVREF <20041231]   # Minister of Refugee Affairs
    FAISAL_AHMAD_SHINWARI_  [AFGGOVJUD <20060931]   # Chief Justice of the Supreme Court
    FARUQ_WARDAK_   [AFGGOVEDU >20081101]   # Minister of Education
    FOREIGN_MINISTER_ABDULLAH_  [AFGGOV <20060531]  # Minister of Foreign Affairs
    GHULAM_MOHAMMAD_AILAQ_  [AFGGOVBUS 20100401 - 20100631] # Minister of Commerce and Industry "Acting"
    GUL_AGHA_SHIRZAI_   [AFGGOVDEV 20031001 - 20041231] # Minister of Urban Development
    HABIBA_SORABI_  [AFGGOVHRI <20041231]   # Minister of Womens Affairs
    HABIBULLAH_GHALEB_  [AFGGOVJUD >20100201]   # Minister of Justice
    HABIBULLAH_QADERI_  [AFGGOVCOP 20050101 - 20070931] # Minister of Counternarcotics
    HAKIM_TANIWAL_  [AFGGOVBUS 20040601 - 20041231] # Minister of Mines and Industries
    HAMIDOLLAH_FARUQI_  [AFGGOVBUS 20090301 - 20100331] # Minister of Transportation "Acting"
    HAMIDULLAH_QADERI_  [AFGGOVBUS 20080401 - 20081231] # Minister of Transportation
    HASAN_BANO_GHAZANFAR_   [AFGGOVHRI >20061001]   # Minister of Womens Affairs
    HEDAYAT_AMIN_ARSALA_    [AFGGOV <20050131][AFGGOVBUS 20050201 - 20060531]   # Vice President & Minister of Commerce
    IBRAHIM_ADEL_   [AFGGOVBUS 20060601 - 20100131] # Minister of Mines and Industries
    ISHAQ_SHAHRYAR_ [AFGGOV <20030931]  # Ambassador to the US 
    ISMAIL_KHAN_    [AFGGOVBUS >20050101]   # Minister of Energy, Water, and Power
    JAMAHIR_ANWARI_ [AFGGOVREF >20100701]   # Minister of Refugees and Repatriation
    JARULLAH_MANSOORI_  [AFGGOVAGR >20100201]   # Minister of Rural Rehabilitation and Development
    JUMA_MOHAMMAD_MOHAMMADI_    [AFGGOVBUS <20030931]   # Minister of Mines and Industries
    KHODAIDAD_  [AFGGOVCOP 20071001 - 20100131] # Minister of Counternarcotics "Acting"
    MASSOUDA_JALAL_ [AFGGOVHRI 20050101 - 20060531] # Minister of Womens Affairs
    MIR_MOHAMMAD_AMIN_FARHANG_  [AFGGOV 20050101 - 20060531]    # Minister of Economy
    MIR_MOHAMMAD_SEDIQ_ [AFGGOVBUS 20050101 - 20060531] # Minister of Mines and Industries
    MOHAMMADALLUH_BATASH_   [AFGGOVBUS 20100401 - 20100731] # Minister of Transportation "Acting"
    MOHAMMAD_ALAM_RAZAM_    [AFGGOVBUS <20041231]   # Minister of Light Industry and Food Stuffs
    MOHAMMAD_AMIN_FARHANG_  [AFGGOV <20041231][AFGGOVBUS 20061001 - 20081231]   # Minister of Reconstruction & Minister of Commerce
    MOHAMMAD_AMIN_NASERYAR_ [AFGGOVMOS <20041231]   # Minister of Hajj and Mosques
    MOHAMMAD_AREF_NOORZAI_  [AFGGOV <20041231]  # Minister of Border and Tribal Affairs
    MOHAMMAD_AZAM_DADFAR_   [AFGGOVEDU 20060601 - 20100331] # Minister of Higher Education
    MOHAMMAD_FAHIM_KHAN_    [AFGGOV <20050131][AFGGOV >20100101][AFGGOVMIL <20041231]   # Vice President & First Vice President & Minister of Defense
    MOHAMMAD_HANIF_ATMAR_   [AFGGOV 20081101 - 20100631][AFGGOVAGR <20060531][AFGGOVEDU 20060601 - 20081031]    # Minister of Interior & Minister of Rural Development & Minister of Education
    MOHAMMAD_JALIL_SHAMS_   [AFGGOV 20061001 - 20100131]    # Minister of Economy
    MOHAMMAD_KARIM_BRAHAWI_ [AFGGOV 20060601 - 20081231][AFGGOVREF 20090101 - 20090631] # Minister of Border and Tribal Affairs & Minister of Refugees and Repatriation
    MOHAMMAD_MASOOM_STANAKZAI_  [AFGGOVMED <20041231]   # Minister of Communications
    MOHAMMAD_MIRWAIS_SADEQ_ [AFGGOVBUS <20040531]   # Minister of Civil Aviation and Tourism
    MOHAMMAD_SARWAR_DANESH_ [AFGGOVEDU >20100401][AFGGOVJUD 20060601 - 20100131]    # Minister of Higher Education "Acting" & Minister of Justice
    MOHAMMAD_SEDIQ_CHAKARI_ [AFGGOVMOS 20090401 - 20100131] # Minister of Hajj and Islamic Affairs "Acting"
    MOHAMMAD_SHAKER_KARGAR_ [AFGGOV <20041231]  # Minister of Water and Power
    MOHAMMAD_SHARIF_FAEZ_   [AFGGOVEDU <20041231]   # Minister of Higher Education
    MOHAMMAD_YOUSEF_PASHTUN_    [AFGGOVDEV 20090301 - 20100331] # Minister of Urban Development
    MOHAMMAD_YUSUF_NIAZI_   [AFGGOVMOS >20100201]   # Minister of Hajj and Islamic Affairs
    MOHAMMAD_YUSUF_PASHTUN_ [AFGGOVDEV <20030931]   # Minister of Urban Development
    MOQBAL_ZARAR_   [AFGGOV 20060601 - 20070331]    # Minister of Interior
    NEMATOLLAH_SHAHRANI_    [AFGGOVMOS 20050101 - 20060531] # Minister of Hajj and Islamic Affairs
    NIAMATULLAH_EHSAN_JAWID_    [AFGGOVBUS 20061001 - 20080331] # Minister of Transportation
    NIAMATULLAH_SHAHRANI_   [AFGGOV <20050131][AFGGOVMOS 20060601 - 20090331]   # Vice President & Minister of Hajj and Islamic Affairs
    NOOR_MOHAMMAD_QARQIN_   [AFGGOVLAB <20041231]   # Minister of Labor and Social Affairs
    NUROLLAH_DELAWARI_  [AFGGOVBUS 20050101 - 20071131] # Governor, Central Bank
    NUR_MOHAMMAD_QARQIN_    [AFGGOV 20060601 - 20100131][AFGGOVEDU 20050101 - 20060531] # Minister of Martyrs, Disabled, and Social Affairs & Minister of Education
    OBAIDULLAH_RAMIN_   [AFGGOVAGR 20050401 - 20081031] # Minister of Agriculture and Food
    OMAR_ZAKHILWAL_ [AFGGOV >20090301][AFGGOVBUS 20090101 - 20090231]   # Minister of Finance & Minister of Transportation "Acting"
    RAHMATULLAH_NABIL_  [AFGGOVMIL >20100801]   # Director General "Acting", National Directorate of Security
    RAMAZAN_BASHARDOST_ [AFGGOV 20040601 - 20041231]    # Minister of Planning
    RANGIN_DADFAR_SPANTA_   [AFGGOV 20060601 - 20100131][AFGGOVMIL >20100601]   # Minister of Foreign Affairs & National Security Adviser
    RAVAN_A.G._FARHADI_ [AFGGOV 20030801 - 20070131]    # Permanent Representative to the UN 
    SAIDTAYEB_JAWAD_    [AFGGOV 20040601 - 20041231]    # Ambassador to the US 
    SAID_TAYEB_JAWAD_   [AFGGOV 20050101 - 20101031]    # Ambassador to the US 
    SARWAR_DANESH_  [AFGGOVJUD 20050101 - 20060531] # Minister of Justice
    SAYED_AMIR_SHAH_HASANYAR_   [AFGGOVEDU 20050101 - 20060531] # Minister of Higher Education
    SAYED_EKRAMODDIN_MASUMI_AGHA_   [AFGGOVLAB 20050101 - 20060531] # Minister of Labor and Social Affairs
    SAYED_HUSSAIN_ANWARI_   [AFGGOVAGR <20030931]   # Minister of Agriculture and Livestock
    SAYED_MAKHDOOM_RAHEEN_  [AFGGOVMED <20030931]   # Minister of Information and Culture
    SAYED_MAKHDUM_RAHIN_    [AFGGOVMED 20050101 - 20060531][AFGGOVMED >20100201]    # Minister of Information and Culture
    SAYED_MOHAMMAD_ALI_JAWED_   [AFGGOVBUS <20030931]   # Minister of Transportation
    SAYED_MOHAMMAD_AMIN_FATEMI_ [AFGGOVHLH 20050101 - 20100331] # Minister of Health
    SAYED_MUSTAFA_KAZEMI_   [AFGGOVBUS <20030931]   # Minister of Commerce
    SEDIQA_BALKHI_  [AFGGOV 20050101 - 20060531]    # Minister of Martyrs and Disabled
    SEYYED_HUSSAIN_ANWARI_  [AFGGOVAGR 20031001 - 20041231] # Minister of Agriculture and Livestock
    SEYYED_MAKHDOOM_RAHEEN_ [AFGGOVMED 20031001 - 20041231] # Minister of Information and Culture
    SEYYED_MOHAMMAD_ALI_JAWED_  [AFGGOVBUS 20031001 - 20041231] # Minister of Transportation
    SEYYED_MUSTAFA_KAZEMI_  [AFGGOVBUS 20031001 - 20041231] # Minister of Commerce
    SEYYED_TAYEB_JAWAD_ [AFGGOV 20031001 - 20040531]    # Ambassador to the US 
    SHER_MOHAMMAD_ETEBARI_  [AFGGOVREF 20071201 - 20081231] # Minister of Refugees and Repatriation
    SIMA_SAMAR_ [AFGGOVHRI <20070531]   # Chair of the Human Rights Commission
    SOHAILA_SEDIQ_  [AFGGOVHLH <20041231]   # Minister of Public Health
    SOHRAB_ALI_SAFARI_  [AFGGOV 20050101 - 20060531]    # Minister of Public Works
    SULTAN_HOSSAIN_HESARI_  [AFGGOVDEV >20100401]   # Minister of Urban Development "Acting"
    SURAB_ALI_SAFARI_   [AFGGOV 20060601 - 20100631]    # Minister of Public Works
    SURAYA_DALIL_   [AFGGOVHLH >20100401]   # Minister of Health "Acting"
    TAJ_MOHAMMAD_WARDAK_    [AFGGOV <20030931]  # Minister of Interior
    USTAD_HAJI_MOHAMMAD_MOHAQQEQ_   [AFGGOV <20040531]  # Minister of Planning
    WAHIDULLAH_SHAHRANI_    [AFGGOVBUS >20090101]   # Minister of Commerce and Industry "Acting"
    YOUSEF_PASHTUN_ [AFGGOVDEV 20060601 - 20090231] # Minister of Urban Development
    YUNIS_QANUNI_   [AFGGOVEDU <20040931]   # Minister of Education
    YUSOF_PASHTUN_  [AFGGOVDEV 20050101 - 20060531] # Minister of Housing and Urban Development
    ZAHIR_TANIN_    [AFGGOV >20070201]  # Permanent Representative to the UN, New York
    ZALMAI_RASSOUL_ [AFGGOVMIL 20050901 - 20060131] # National Security Adviser
    ZALMAY_RASSOUL_ [AFGGOV >20100201][AFGGOVMIL 20031201 - 20050831][AFGGOVMIL 20060201 - 20100131]    # Minister of Foreign Affairs & Presidential Adviser on National Security & National Security Adviser
</Government>
</Country>
1

There are 1 best solutions below

2
Michael Kay On

There are a number of things that make the file ill-formed XML.

  • It starts with a big comment, which includes occurrences of "--" which is not allowed in XML comments
  • There is no root element that encloses everything else
  • It includes elements that have spaces in the name:
    <Chairman of the military council>
            ABDUL_QADIR_    [19780427 - 19780430]  [B:19440101]
    </Chairman of the military council>

I couldn't see any problem with the NATIONALITY attribute, however.

It's hard to see why anyone would choose to publish data like this and then make it so difficult to read, other than sheer incompetence. They have obviously put a lot of effort into managing the publication process and cataloguing all the metadata, so it's very strange that they should deliver something so unprofessional.

If this is a one-off then I would edit it by hand to turn it into well-formed XML. If it's something you need to do repeatedly then I would contact the supplier of the data to ask them what they think they are playing at and whether they have an alternative format available. (Don't be afraid to do this; I've done it in the past and found suppliers very grateful for the feedback.)

You could try putting it through a tool like TagSoup to see if that makes any sense of it.