Linux

Bad advise for sed. regex101 doesn't support POSIX regexes, so you are unable to get the same results as with sed.

Hello :) Thanks for your reply !

That's exactly what I did and how I came to my "final" result but I doesn't work as expected... because the lack of knowledge and understanding !

Will give sd a try and see if I can come up with something ! Thanks for the pointer !

Yeah probably bare bone regex was a mistake however a friendly user gave me a step by step guide on how to achieve my goal:

#! /bin/bash

files="/home/USER/projects/test.md"

mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"

while IFS= read -r line; do
	#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9]) 
	dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
	sed -i "s/$line/${dashlink}/" "$files"

	#Puts everything to lowercase after a hashtag
	lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
	sed -i "s/$dashlink/${lowercaselink}/" "$files"

	#Removes spaces (%20) from markdown links after a hashtag
	spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
	sed -i "s/$lowercaselink/${spacelink}/" "$files"

done <<<"$mdlinks2"

If you know a better way to achieve similar results I'm very open for every new lead and learn something new !

Hello :) Sorry for the very late response !

Effectively your regex is very close as a one line, I'm pretty impress ! :0 However I missed to mention something In my post (I only though about it after working on it with another user in the comments...). There a 2 things missing on your beautiful and complex regex:

1. Numbering with dots also needs to have a dash in between (actually I think every special characters like spaces or a dots are converted to a dash )

FROM
---------------
[Link with numbers](readme.md#1.3%20this%20is%20another%20test)

TO
---------------
[Link with numbers](readme.md#1-3-this-is-another-test)

2. The part before the hashtag needs to keep it original form (links to a real file)

FROM
---------------
[Link with numbers](Another%20file%20to%20readme.md#1.3%20this%20is%20another%20test.md)

TO
---------------
[Link with numbers](Another%20file%20to%20readme.md#1-3-this-is-another-test.md)

Sorry for the trouble I wasn't aware of all the GitHub-Flavored Markdown syntax :/. I got a a very cool working script that works perfectly with another user but If you want to modify your regex and try to solve the issue in pure regex feel free :) I'm very curious how It could look like (god regex is so obscure and at the same time it has some beauty in it !)

#! /bin/bash

files="/home/USER/projects/test.md"

mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"

while IFS= read -r line; do
	#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9]) 
	dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
	sed -i "s/$line/${dashlink}/" "$files"

	#Puts everything to lowercase after a hashtag
	lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
	sed -i "s/$dashlink/${lowercaselink}/" "$files"

	#Removes spaces (%20) from markdown links after a hashtag
	spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
	sed -i "s/$lowercaselink/${spacelink}/" "$files"

done <<<"$mdlinks2"

annotated it is working like this:

# use a loop to iteratively replace the %20 with -, since doing s/%20/-/g would replace too much. we loop until it cant substitute any more

# label for looping
:loop;
# skip the following substitute command if the line contains an http link in markdown format
/\[[^]]*\](http/!
# capture each part of the link, and join it together with -
s/\(\[[^]]*\]\)\(([^)]*\)%20\([^)]*)\)/\1\2-\3/g;
# if the substitution made a change, loop again, otherwise break
t loop;

# convert all insides to the link lowercase if the line doesnt contain an http link
/\[[^]]*\](http/!
# this is outside the loop rather than in the s command above because if the link doesnt contain %20 at all then it won't convert to lowercase
s/\(\[[^]]*\]\)\(([^)]*)\)/\1\L\2/g

Hello,

I have thought of a python script and looked a bit around but couldn't find something satisfactory. Also I'm a tiny bit more versed in bash/CLI than with python... Even though that's very arguable !

I looked through the Github repo and at first glance I have no idea how this could do the job, again I probably have to dig a bit deeper and understand what this is actually doing !

Thanks for the pointer will give it a try :)

Thank you very much for taking your time and trying to help me with comments and all !

you need a full featured markdown parser for this.

Do you mean something like pandoc? Someone pointed me to it and it seems it can covert to GitHub-Flavored Markdown ! Thanks for the pointer will give it a try to see how it works out with my actual script :)

Sorry for the very late response !! Here is the working bash script another user helped me put together:

#! /bin/bash

files="/home/USER/projects/test.md"

mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"

while IFS= read -r line; do
	#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9]) 
	dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
	sed -i "s/$line/${dashlink}/" "$files"

	#Puts everything to lowercase after a hashtag
	lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
	sed -i "s/$dashlink/${lowercaselink}/" "$files"

	#Removes spaces (%20) from markdown links after a hashtag
	spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
	sed -i "s/$lowercaselink/${spacelink}/" "$files"

done <<<"$mdlinks2"

NB: global substitution s///g is not applicable here because you need to perform new substitutions in a substituted text. Both sed regexp syntaxes (basic and extended) don't support lookarounds that could solve this issue.

Sorry for the late response... I was busy with another user :S My English is so bad I'm not able to response to every one at the same time... Whatever...

I tried your pearl regex substitution and effectively it does what I ask from my post, so thank you very much for your help ! However, I missed a few use cases were your regex breaks... But that's on me, your command works as expected !!!

[Link with numbers](Another%20Markdown%20file.md#1.3%20this%20is%20another%20test.md)

The part before the hashtag need to keeps it's original form (even with %20) because it links to a markdown file directly and not a header (Hope it's comprehensible?). It took me a lot of time with another user and we came to a wrapped up script that does everything:

#! /bin/bash

files="/home/USER/projects/test.md"

mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"

while IFS= read -r line; do
	#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9]) 
	dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
	sed -i "s/$line/${dashlink}/" "$files"

	#Puts everything to lowercase after a hashtag
	lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
	sed -i "s/$dashlink/${lowercaselink}/" "$files"

	#Removes spaces (%20) from markdown links after a hashtag
	spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
	sed -i "s/$lowercaselink/${spacelink}/" "$files"

done <<<"$mdlinks2"

If you are motivated you can still improve your regex If you want :) I'm kinda curious If it's possible with a one-liner ! Thank again for your help and sorry for the late response !!

I didn't test this, but it will change the whole URL while changes are only needed in its fragment component (after the first #).

Hello :) Sorry for the late response !!! I was busy working it out with another user ! However out of curiosity gave your sed regex a try, but there seems a missing ( somewhere ! I tried to fix the issue but your regex is way over my capabilities ! If you are sed/regex fanatic a want to give it another try feel free :). Right now I found a solution with another user that works great here's the script in question if you are interested:

#! /bin/bash

files="/home/USER/projects/test.md"

mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"

while IFS= read -r line; do
	#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9]) 
	dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
	sed -i "s/$line/${dashlink}/" "$files"

	#Puts everything to lowercase after a hashtag
	lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
	sed -i "s/$dashlink/${lowercaselink}/" "$files"

	#Replace spaces (%20) from markdown links to - after a hashtag
	spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
	sed -i "s/$lowercaselink/${spacelink}/" "$files"

done <<<"$mdlinks2"

It's not very elegant but it does the job... While working on it with another very friendly user I came across other thing I haven't though of like:

• Converting 1.2 to 1-2 (e.g. [Just a placeholder](#1.2%20Just%20a%20link%20to%20header))
• Linking to another markdown file (e.g. [Just a placeholder](Another%20File.md#1.2%20Just%20a%20link%20to%20header))
• The link to file before the # need to keeps it's original form (e.g. [Just a placeholder](Another%20File.md#1-2-just-a-link-tp-header))

Well I think that bare bone sed/regex wasn't the right tool, but in a bash script it does exactly what I'm expecting :)

Thanks for your help and pointers !

Hello :) Sorry to pin you, I just gave pandoc a try but it doesn't work and I had to dig a bit further into the web to find out why !

Links to Headings with Spaces are not specified by CommonMark and each tool implement a different approach... Most replace space with hyphens other use URL encoding (%20). So even though pandoc looks awesome it doesn't work for my use case (or did i miss something? Feel free to comment).

You can give it a try on https://pandoc.org/try/ with commonmark to gfm:

[Just a test](#Just a test)
[Just a link](https://mylink/%20with%20space.com)
[External link](Readme.md#JUST%20a%20test)
[Link with numbers](readme.md#1.3%20this%20is%20another%20test)
[Link with numbers](Another%20file%20to%20readme.md#1.3%20this%20is%20another%20test)

If you prefere a cli version:

pandoc --from=commonmark_x --to=gfm+gfm_auto_identifiers "/home/user/Documents/test.md" -o "pandoc_test.md"

Thanks for the pointer I wasn't aware pandoc was able to do that :/ It seems It can convert to Github-Flavored Markdown !! I have to give it a try :) Still I learned a lot from another user about regex/sed and Pearl :) !