Home Wiki Blog Support Contact Us
› ›

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

In this Discussion

osTicket v1.10 (stable) and Maintenance Release v1.9.15 are now available! Go get it now

Parsing form variables and data from email bodies to fill out ticket details automatically.

ScientistGuyScientistGuy
edited September 2017 in Mods and Customizations
I recently took over a project from another group, with one of the tasks being to update the current in place osTicket(v1.10) to allow for tickets to be automatically generated upon receiving emails. This part was not so difficult, with the mail fetching already in place. However, I have not been able to populate form variables with data from the email body, it is all currently just dumped into the ticket thread(vars['message']). Has anyone written a block and/or mod that does this and actually works? I have only found one example, and it doesn't work.

Note: The problem is not parsing the email body, the problem seems to lie in either getting the correct form id and/or field names. Also $topic does not seem to be set, but even forcing a value to it doesn't seem to work.

Thank you in advance.
Tagged:

Comments

  • ntozierntozier
    Populating form variables from email is not a feature in osTicket.  You would have to write some sort of parsing routine to do it.  Maybe @mfelber or @Grizly will be able to assist you.
  • ScientistGuyScientistGuy
    I know it isn't currently native to osTicket, I have already started writing a routine into class.mailfetch.php to attempt it. As of yet I have been unsuccessful. But I can't be the only person to have ever attempted this, I'm sure someone out there somewhere has figured it out. 

    using preg_match_all I can easily parse out the email, and from that easily parse out the field variable and its value into an associative array. My issue is actually getting those values into the form. 
  • GrizlyGrizly
    I'm in the same boat mate, can easily manipulate the email, however the form system is still a mystery. I don't really have time to learn it, but if you figure it out, or if the Devs decide to document it.. I'd love to know how.
  • mfelbermfelber
    edited September 2017
    I'm afraid I can't help with that.
  • Micke1101Micke1101
    Would you want to be able to set specific form based on one of these variables?
    Is regex the preferred way of determining where to fetch the data and to what form variable?
  • ScientistGuyScientistGuy
    Currently this is what I have done with what I was given:

    First an email is generated by a form clients fill out, form is set up so that each entry matches a form field. All this was in place before I took over the project. I can manipulate the email that's generated however I wish.

    As I am doing all this on dev server I can manipulate it as I see fit, currently my test emails are in a list format: field=value. I preg match all on vars['message'] with regex to separate out the field labels and the values for the field labels. Labels in the emails are the variables I physically set for the form('orgname' for example). All of this works perfectly, setting a field name = value in associative array. 

    Where it all falls apart is after this. If I try to ensure the vars['topicID'] matches $topic, it fails because $topic turns out to be empty. Even if I ignore that(in this case I only have 1 help topic so it should default to that one anyway), when I attempt to get the form_id(which in this case id=1) it fails because it appears that $title->ht['form_id'] returns 'h' for some reason unknown to me. As such it appears I am unable to access either the form name or the fields in the form.

    what I would like to be able to do is get to the list of the form variables in an array $formVariables and use if(array_key_exists($formVariables,$fullSetValues)){ pass values into $fields} to fill out the ticket details. 
  • GrizlyGrizly
    Just had a sneaky idea, you've got everything you need, can you add a localhost API key then pipe all the details to that?
  • ScientistGuyScientistGuy
    Oh that is sneaky. Let me look into that and I will get back with you
  • Micke1101Micke1101
    Expiremented a little with the plugin concept: https://github.com/Micke1101/OSTicket-plugin-Emailform

    You select what form you want to attach to emails (unfortunately you've to reopen the plugin after saving that for it to trigger the remaining options)

    Then it'll list all the fields available in that form and let you enter regex patterns on each row.

    Then when a email creates a ticket, it'll check if there's a match and if there is it'll populate that field with the data that it finds.
  • GrizlyGrizly
    @Micke1101 that looks really cool. :-)
  • ScientistGuyScientistGuy
    NOW that, is what I'm talking about. That's exactly what I was trying to do, but you have made it far far more user friendly. Works just about perfectly. The only thing I'm going to look into changing up a bit is the thread bit. We have a summary attached to our submissions and I would like for it to go into the thread part, the vars['message'] variable where the email body is saved, and display that on our tickets instead of the email body itself. 

    But this plugin works brilliantly, thank you sir!
  • Micke1101Micke1101
    Thank you :).

    Great to hear.
  • ScientistGuyScientistGuy
    Well it is like halfway there, it connects to the form and does assign values but now I can't seem to get the regex correct, which is really frustrating ha.
  • Micke1101Micke1101
    Do you have an example to work with?
  • ScientistGuyScientistGuy
    I do, give me a little bit to get back to my computer. My guess is it's something to do with the email body itself not being plain text maybe. The email is a list format, field=variable, like First Name = Generic Person Name, on different lines. I can't seem to find a line break and it is saving the whole string after the equals sign. A Regex example I've tried is something like fname=([^=\r\n]+) from memory, I'm on phone but will give accurate assessment when I get back to computer.

    Note, I am not well versed in Regex so it is good chance just me having won't capture syntax. But I tested it separately in eclipse and it worked perfectly so I dunno.
  • GrizlyGrizly
    Explained: https://regexper.com/#fname=(%5B%5E%3D%5Cr%5Cn%5D%2B)%20

    You're better off starting simple, like: fname=(.+)
    But you'd have to change the plugin to work with capture groups.
    https://github.com/Micke1101/OSTicket-plugin-Emailform/blob/9421e19c3a92c41b61a1fa10d9dd18a88bcdf43d/class.EmailformPlugin.php#L62
    Change the $match[0] to $match[1]

  • Micke1101Micke1101
    Try the following pattern instead: (?<=fname=).*
  • ScientistGuyScientistGuy
    I will give that one a whirl too. There's some weird stuff going on and I'm not exactly sure what it is. with the regex fname=([^\r\n]+) it finds the eol, however for lines like fname=some dudes name, it saves the whole string as fname=some dudes name, instead of just some dudes name. I have tried all indexes of the array, they are all the same. Get's weirder when its a line like phone=1111111111 using phone=([^=\r\n]+) it captures only 1111111111 and not phone=1111111111. So still working through it ha
  • ScientistGuyScientistGuy
    Update:

    I found the issue and it has been corrected, it wasn't in the plugin but my own momentary stupidity. The plugin works near perfectly. The only issue with it I have is being able to pass a summary text from the email into the thread, instead of having the email body in the ticket thread. I know what needs to happen, the trick will be finding a way to pass whats extracted from the email into $vars['message']. As of yet I have not found a way to cleanly do this. Any suggestions?
  • ScientistGuyScientistGuy
    I have been testing this plugin for a week or so now and have really found it helpful. I have however found a few issues that may need to be addressed. One appears to be a bug and the other is more with functionality extension. Not sure if its a bug or what but it is very strange. 

    I have 2 fields that used to work perfectly like everything else, however recently when I was attempting to change regex expressions to a different form, these two fields now will no longer save. Whenever I put an expression in, click save it comes up and shows everything saved properly. But when I reload the page or go to a different page and go back the fields are empty again. I don't really have an explanation for this. I have tried deleting the fields and remaking them but the same thing occurs. 

    Another thing is that section breaks do not appear on the ticket. I do not know if this is something with the plugin or with a custom form. Other the other forms I have used section everything out as it's supposed to but the custom forms seem to ignore section breaks.

    The last thing is the email body itself. It does not appear that the string has been decoded. The plugin works perfectly if the email is plain text, but if it isn't I think it breaks my regex.


    Previously I used very simple regex and emails to test and refine the plugin(using email lists styled with field=variable and regex of field=([^=\r\n]+) and it worked perfectly). I noticed the list must be in perfect order with the form fields for it to find matches. I think this can be corrected using preg_match_all instead of preg_match, though I haven't messed with that yet. 

    The email format however seems to break my new email list and regex. This is the form that the plugin in production will have:

    Field name: \n \t variable answer
    First Name:
    John Doe

    I have tested the regex outside of osticket and it works perfectly on field/end of line/tab/variable. However it does not work parsing an email in osticket, even if email is plain text before it is sent. 

    I will continue to work on this but if anyone has any thoughts/insight/ideas that would be Great.
  • ScientistGuyScientistGuy
    Edit:

    First Name:
    John

    Last Name:
    Doe
  • mfelbermfelber
    ?
  • GrizlyGrizly
    It's because of the dynamic plugin config.. that never worked for me. Attempting to change a plugin's fields config in any dynamic fashion resulted in config corruption for the plugin. Not sure why. Probably should report it as a bug to osTicket, however I'm not sure if it is a bug, or just an assumption that we will use a static config for a plugin.
    You might get away with it by saving the config form twice.. however it's not guaranteed.. 

  • ScientistGuyScientistGuy
    I believe deleting it and resetting everything anew will fix it, I will try to just not edit it too much which shouldn't be a problem once all kinks are worked out and moved to prod. Note saving twice appears to have no effect.
Sign In or Register to comment.