Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

In this Discussion

osTicket v1.10 (stable) and Maintenance Release v1.9.15 are now available! Go get it now

Some (~5%) emails are not fetched

Hello,

I am experiencing a weird situation. Around 5% of the e-mails are not fetched. After some struggling I have setup fetching e-mails from the server and also sending e-mails (needed to modify stmp.php file).

So I was finally satisfied, when I discovered (after complains) that some e-mails are not fetched. Possibly also some e-mails are not sent, but this I cannot reall check.

OSTICKET
Version 1.9.14 8b927a0
osTicket Version     v1.9.14 (8b927a0) — Up to date
Web Server Software     Apache
MySQL Version     5.5.49
PHP Version     5.6.21
PHP Extensions: gdlib, imap, xml, xml-dom, json, mbstring, phar, fileinfo
PHP Settings: cgi.fix_pathinfo: "1"
OSTICKET is setup on synology (but not from an app installer, installed manually)

EMAIL SETUP:
Fetching:
Status: Enable
Hostname: imap.1und1.de
Port Number: 993
Mail Box Protocol: IMAP + SSL
Fetch Frequency: 15min
Emails per Fetch: 50
Fetched Emails: Do nothing
Sending:
Status: Enable
Hostname: smtp.1und1.de
Port: 587
Auth: Yes
Header Spoofing: Allow
Just to emphasize this is confirmed to work, so setup is rather out of suspicion.

Incoming Email settings:
Email Fetching: Enabled, Fetch on auto-cron
Accept All Emails: Yes, from uknown too

Report
After complains due to no answer for some tickets I have entered the inbox of the e-mail and started to manual comparing what was fetched by osTicket and what was not. Unfortunately there was no conclusion. Further in the description I will refer to the emails that were not fetched as "the emails".


Troubleshooting:
1. First I though it is during some server update or backup (it is installed on synology), but since it is done in a scheduled matter and the emails' timing were not aligning, I have rejected this theory
2. Then I though it is some banned users or spammers list, but then I discovered it happened many times that one user could send first e-mail, create a ticket get couple of replies and suddenly one e-mail was not fetched. Later the same user could create another ticket. Therefore it could not be related to e-mail address.
3. Then I though there is some (i.e.) signature with full html/js that is being rejected, but again with one user I have done some troubleshooting and he was sending all e-mails from the same PC, same outlook, same signature. One was fetched, another not. So probably not html/js in content
4. I have tried to put back the emails to the osticket by forwarding them (inbox opened as folder in my outlook) then forward, remove my signature, set "reply to" to the address of the person and forward. Then e-mail would pop up in the inbox as continued discussion and hopefully it would be fetched.
4a. while forwarding I have tested this method several times and it worked well, but not always! Of course I was mostly forwarding e-mails that were not fetched, so I think that high rate of failure was related to e-mails
4b. I have been modifying emails in various way to test forwarding/fetching. For example I change format html->rich text / html->text or removed signature or even topic or further modifications. Unfortunately I could not figure out a pattern here. Some e-mail were successfully fetched after forwarding, MOST not.

I have now collected all the emails in a special folder for checking similarities. If the failed forwards should be excluded there are now 11 emails for last 2 months. I have even checked headers, but since I do not know what to look for, I could have missed something valuable.

I would appreciate your hints in solving the issue. You may imagine it is quite annoying to have weekly review if there were any emails not fetched to add them manually.


Tagged:

Comments

  • edited August 2016
    Disable auto-cron and setup a cron or windows task scheduler job to run cron.php.  Usiung autocron is not recommended and is only intended to be used when there is no other option.

    Do you have Admin panel -> Settings -> Access -> Registration Required: checked?

    Personally I would lower fetch frequency to 2 minutes, and decrease Emails per fetch to 10.

    Do you have any ticket filters?  (try disabling them)
  • Hi,

    Thanks for the interest.

    I have disabled the auto-cron and setup one in synology:
    Control Panel \ Task Scheduler: run by "admin; scheduled every 5 minutes from 6:00-23:00 to run, but I am afraid it is now working. I consider some of the following errors might be related to synology setup, but nevertheless, maybe someone is able to help. I have executed this manually from ssh and it came back with:

    PHP Warning:  include(): Unable to find the wrapper "phar" - did you forget to enable it when you configured PHP? in /volume1/web/support/include/class.plugin.php on line 280

    So I have fixed it with adding the phar lib, running as:
    sudo /usr/local/bin/php56 -d extension=phar.so /usr/syno/synoman/phpunit-5.5.2.phar /volume1/web/support/api/cron.php
    the outcome was:
                                                                             
    PHP Warning:  file_exists(): open_basedir restriction in effect. File(phpunit.xml) is not within the allowed path(s): (/etc.defaults:/etc:/usr/syno/synoman:/var/services/tmp:/var/services/web:/var/services/homes:/volume1/@appstore/Moodle:/volume1/@tmp/php:/volume1/moodle/:/volume1/web:/volume1/web/support/api/) in phar:///usr/syno/synoman/phpunit-5.5.2.phar/phpunit/TextUI/Command.php on line 699
    PHP Warning:  file_exists(): open_basedir restriction in effect. File(phpunit.xml.dist) is not within the allowed path(s): (/etc.defaults:/etc:/usr/syno/synoman:/var/services/tmp:/var/services/web:/var/services/homes:/volume1/@appstore/Moodle:/volume1/@tmp/php:/volume1/moodle/:/volume1/web:/volume1/web/support/api/) in phar:///usr/syno/synoman/phpunit-5.5.2.phar/phpunit/TextUI/Command.php on line 701
    PHP Fatal error:  Call to a member function logDebug() on null in /volume1/web/support/include/api.cron.php on line 21

    So at the moment I am not sure if this is synology issue or a fault in script, but I cannot execute it neither via cron nor manually.

    Also I have plenty filters and it took me a while to set them up, so i am not going to disable them. I forgot to mention that i have also reviewed the log in osticket to check if it is not some filter, but not evidence that any of the tickets above was rejected due to filter setup.

    Anyway, is it clear for you that this skips in fetching is a result of auot-cron? Solving this cron seems to be achievable and possibly not related to osTicket. How come auto-cron can skip e-mails? Is it some sort of overload?
  • PHP 5.4+ has phar enabled in it by default.  So if your getting an error about phar not working then there is a problem with your PHP.

    This "open_basedir restriction in effect" means that your PHP is configured using open_basedir and it is preventing users from opening files and scripts outside of the base dir you specified in your php.ini.

    You should address the issues with PHP prior to us continuing to troubleshooting your issue.
  • Hi ntozier,

    Thank you for help. I will follow up this issue first with synology. Although I might come back ;)
  • Hi, so the synology support sucks. My questions were ignored. Although I still have some doubts how this can be related to the auto-cron.

    Consider this situation:

    I find an e-mail that is not fetched. I forward via external server and after some time appears in Inbox. By osTicket is should be considered as new e-mail, but still it is not fetched. I continue doing this, I forward the same e-mail again and again, now Inbox has several copies of the same e-mail, but osTicket keeps ignoring all of them. Then I just click "new message" and send it to the osTicket e-mail and it is being fetched. Then I continue forwarding the old message and osTicket is not fetching it.

    So considering that all of the e-mails among period of time (new message and forwarded) are sent from the same e-mail address and only one gets to osTicket I assume there is something wrong in the content of the e-mail that was not fetched.

    I seriously cannot see this being related to cron.


  • This response is kind of all over the place.

    I told you earlier in this thread to disable auto-cron in osTicket and to setup your own cron job.
    You did this and ran into an issue phar not being enabled. You did something to enable phar and ran into an issue with"open_basedir restriction in effect." I'm not sure why or how your version of PHP (5.6.x) has phar disabled since it has been part of PHP since 5.3+.  Either your not running 5.6.x or you enabled it incorrectly.


    That is what you need to resolve. You may need to talk to Synology support to resolve this.  My guess is that your synology is running more than one version of PHP.
Sign In or Register to comment.