how to split ldif in smaller files using python?

Question

Given a ldif file with 999 of records to be updated with ldapmodify, how can i split it in smaller files?

---

https://stackoverflow.blog/2011/07/01/its-ok-to-ask-and-answer-your-own-questions/

Answer 1

Executing splitLdifByFiles('/tmp/bigfile.ldif', 10) will create 10 files and evenly distribute all records between them in a round robin manner.

def splitLdifByFiles(ldiffile, nfiles):
    '''
    Recebe um arquivo ldif <ldiffile> e separa em N arquivos <nfiles> menores (round robin)
    '''

    with open(ldiffile, 'r') as fr:
        reader = fr.read()
        chunks = ['\n'+c+'\n' for c in reader.split('\n\n') if c]

    files = [open('/tmp/df.chunk.%d.ldif' % i, 'w') for i in range(nfiles)]

    for idx, chk in enumerate(chunks):
        files[idx % nfiles].writelines(chk)

    for f in files:
        f.close()

Answer 2

Executing splitLdifByRecords('/tmp/bigfile.ldif', 100) will write 100 records to the output file and close it, repeating this process until it reaches the last chunk of data.

In this example, the last file will contain the remaining 99 records.

def splitLdifByRecords(ldiffile, maxrecords):
    '''
    Recebe um arquivo ldif <ldiffile> e separa em N registros <maxrecords> por arquivo
    '''

    with open(ldiffile, 'r') as fr:
        reader = fr.read()
        chunks = ['\n'+c+'\n' for c in reader.split('\n\n') if c]

    chunkfile = None

    for idx, chk in enumerate(chunks):
        if idx % maxrecords == 0:
            if chunkfile: chunkfile.close()
            chunkfile = open('/tmp/df.chunk.%d.ldif' % (idx + maxrecords), 'w')
        chunkfile.writelines(chk)

    if chunkfile: chunkfile.close()