Script "delete_olditems.py" broken with recent __init__.py

[umgfoin]

"for folder in user.store.folders(parse=True):" seems to have a problem:
...
"getMailAge(item.prop(PR_MESSAGE_DELIVERY_TIME).mapi_value.unixtime)" will fail, either.

Example:
[root@root scripts]# python ./delete_olditems.py --folder="postgres" --user="test" 10 --verbose
Running for user: test
Traceback (most recent call last):
File "./delete_olditems.py", line 57, in
main()
File "./delete_olditems.py", line 42, in main
for folder in user.store.folders(parse=True):
File "/usr/lib64/python2.6/site-packages/zarafa/init.py", line 1270, in folders
yield self.folder(path.decode(sys.stdin.encoding)) # XXX can optparse output unicode?
File "/usr/lib64/python2.6/site-packages/zarafa/init.py", line 1247, in folder
return self.subtree.folder(key, recurse=recurse, create=create)
File "/usr/lib64/python2.6/site-packages/zarafa/init.py", line 1596, in folder
raise ZarafaNotFoundException("no such folder: '%s'" % key)
zarafa.ZarafaNotFoundException: no such folder: 'postgres'

[srepmub]

do you mean that previously, when a folder given on the command-line does not exist for a user, it would be skipped, but now you get an error..?

the variable 'mapi_value' was removed some time ago, and should probably be changed to 'mapiobj.Value'. not sure why we would actually need to look at PR_MESSAGE_DELIVERY_TIME though, since we have item.received.

[umgfoin]

Sorry, I was not clear enough on this:
In my example, the folder "postgres" exists on the server, nevertheless, the above error is thrown. The script works well with init.py taken from ZCP 7.2.1.51665
If no foldername is specified, the script works fine, too.

The problem with "mapi_value.unixtime" applies only if option --verbose is used. I'm not too deep into MAPI, but the usage of mapi_value.unixtime might have to do with getMailAge is expecting a timestamp (instead of a datetime value).

The folderfilter-problem could apply to other scripts, too - I haven't verified this, so far.
bg

[srepmub]

so the folder is probably not directly under the 'subtree'? I guess the problem is that we changed the command-line parsing so you now have to specify a full 'path'. so for example if the folder is under 'Inbox', you have to specify something like --folder="Inbox/postgres". of course the Inbox itself can still be specified as ---folder="Inbox". hope this is not a problem for your use case?

as for getMailAge, we now convert the timestamp to datetime, but the datetime is already available as item.received.. so I think it's the same as:

getMailAge(item.received)

def getMailAge(messagetime):
# skip first line, as we already have messagetime

[umgfoin]

...understand, yes, in this case it's "/Posteingang/Mailings/Postgres"
The problem with fullpath-addressing is that the path-part for system-folders (as inbox) is localized within zarafa: By default, the script searches named folders in every user's mailbox.
Thus, we e.g. have mailbox-structures with "Inbox/subfolder" and "Posteingang/subfolder". As the script stops now with every zarafa.ZarafaNotFoundException, we effectively can't trim subfolders for every user, even if we do this in separate runs addressing different localizations of "Inbox".
IMHO, two changes would be required to restore compatibility with former versions:

• Let script continue (with other users's mailboxes), if a named folder can't be localized within the current mailbox-tree (try/except around "for folder in user.store.folders(..."
• Have user.store.folders(parse=True) use unique aliases for system-folders

[srepmub]

sorry for the late reply.

what would you think of just giving the english names (such as "Inbox") special meaning? so for example, for "Inbox", we take store.inbox and so on.

"Posteingang" will still work then if there's a "Posteingang", but it won't be localized.

[umgfoin]

You mean, "/store.inbox/subfolder" would act as an alias for "/Posteingang/subfolder", "/Inbox/subfolder", "/Indbakke/subfolder" etc. ?
Or something like
for item in user.store.inbox.items() [...]
Or filtering with "Inbox" would find both "Posteingang" and "Inbox"? I think, the latter was the behaviour in earlier python-zarafa implementations.

[srepmub]

yeah, the latter. it would find "Posteingang" or "Inbox" when filtering on "Inbox". so the name "Inbox" gets a bit of a special meaning.

in previous implementations, it would only find "Inbox" I guess, but you were looking for "postgres" so it didn't matter what the inbox was called.

[jelly]

I just re-factored the script to fix the first error and cleaned up the code.

It seems that we can improve the output a lot, it might be interesting to list the free'd size and number of items. Example output:

==> finished dry run: 747 candidates (disk space saved: 4.8 GiB)

We can probably also improve the performance, if we don't remove item per item, but a list of items per folder.

I hope to discuss a fix for the folders with @srepmub monday ;)