A quick Google suggests what you have.
If the code you have quoted is verbatim what you have tried, seems like you need to extract the parentheses and possibly a single or double quote, depending on the source css. The example source you have given has a single quote.
select-before(select-after(//div/@style, "backgound-image: url("), ")")
Should be (notice the extra '
relating to url('...url')
)
select-before(select-after(//div/@style, "backgound-image: url('"), "')")
But I don't think that would cause xpath to fail... It would just extract the wrong value
Edit:
Further reading suggests xpath 1.0 does have limited functionalities. But, like you, can't find anything concrete.