Programming

17661 readers

216 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]

founded 2 years ago

MODERATORS

snowe

Ategon

[email protected]

I just spent 6 hours of debugging some code, just to find out I had to delete a line in a .gitkeep file (github.com)

submitted 2 months ago* (last edited 2 months ago) by chraebsli to c/programming

8 comments fedilink hide all child comments

I just spent 6 hours of debugging some code, just to find out I had to delete a line in a .gitkeep file:

I'm developing a Grav skeleton, which is a kind of template for Grav CMS to easily create sites with it. Such skeletons can include themes and plugins, which can be used to enhance its features and make individual sites.

Now I had this error that the sitemap.xml (used to tell search engines which pages should be indexed and where to find them) threw an error because the XML declaration wasn't on the first line as it should, but on the 2nd line. It came out that every page had this empty line, no matter if html, json or xml.
To find the bug causing this, I first disabled all plugins and enabled them step by step to find out which one produces said bug. It was the shortcode-core plugin, which can be used to style text with more than just markdown. Since this plugin isn't related in any way to the sitemap plugin, I searched for other possible error sources.
After some playing around, I found out that when I remove a theme, the bug simply disappears. With that info, it had to be a problem with said theme. Since themes can include custom shortcodes and the shortcode plugin did make some "noises", I started investigating a bit more with that. While trying to find the bug, I accidentally deleted the shortcode folder, which had only a .gitkeep file in it. This file was necessary that the folder persist with git versioning because if there was no such directory, the plugin would output an error message. I then recreated the folder manually and the error was still gone. Even after recreating the .gitkeep file, there was no empty line in the output.
After trying to find the bug, I checked git to see which edit made the bug resolve. And there was it: the .gitkeep file I recreated had some changes? But I didn't make any changes to the file except for recreating it. Wrong. The file originally had 2 empty lines, probably accidentally. And by recreating the file, it only had one line which resolved the bug.

It came out that somehow the shortcode plugin tried to get shortcodes from this file and likely added the 2 empty lines to the output. And an empty line before the start of the XML declaration causes the XML parser to output an error. Deleting the 2nd empty line from the .gitkeep file removes the empty line from the output and the error is gone. Bug fixed.

And I spent 6 hours today with the help of a kind stranger in a discord channel who was a bit like a 2nd brain to me. It really helped having someone who hasn't worked on this skeleton and could see some bugs you don't see because you're too familiar with it ("Betriebsblind" in German).

Edit: The plugins don't get maintained by me, they're getting maintained by the Grav team. I plan to create an issue on GitHub if I know more and probably identified the code causing the problem.

top 8 comments

sorted by: hot top controversial new old

[–] [email protected] 26 points 2 months ago (1 children)

You found an important bug in your short code plugin. Removing the line from .gitkeep is not actually the solution; it was a symptom of a much bigger and more dangerous problem: you are inadvertently including and parsing a file that is not intended to be a short code.

You, or a crafty hacker, might one day create a file with code in it that should not be parsed as a short code, and not realize that it’s being done. You’re lucky that you’re the one who discovered this and not somebody else.

The solution is the only parse the files that you need to parse. This means ignoring hidden files that begin with a dot. You might also think about creating a default ignore list for any other non-shortcode file that could exist.

[–] chraebsli 1 points 2 months ago (1 children)

Every shortcode is a PHP class and looks like this:

<?php

namespace Grav\Plugin\Shortcodes;

use Grav\Common\Utils;
use Thunder\Shortcode\Shortcode\ShortcodeInterface;

class ClearBothShortcode extends Shortcode
{
    public function init()
    {
        $this->shortcode->getHandlers()->add('clearboth', function (ShortcodeInterface $sc) {

            // Get shortcode content and parameters
            $str = $sc->getContent();

            $output = '<div style="clear: both;"></div>';

            return $output;

        });
    }
}

with that, it should be possible to only call these classes.

[–] thesmokingman 6 points 2 months ago (1 children)

I agree with comment OP; you haven’t solved the problem. The number of empty lines in a file that shouldn’t be parsed shouldn’t affect your code. If it is, then you need to stop parsing files that shouldn’t be parsed. For example, if this arbitrary file is being included (totally valid assumption given your debugging), what’s to prevent a malicious payload from being included or executed?

I genuinely have no idea how a random text file, much less a dot file, gets parsed in a PHP project. It feels like there’s no attempt at file validation which is really fucking important for server-side code.

[–] chraebsli 1 points 2 months ago (1 children)

Hey thanks for the comment!
The plugins don't get maintained by me, they're getting maintained by the Grav team. I'm not very good in PHP too.
I plan to create an issue on GitHub if I know more and probably identified the code causing the problem.

[–] thesmokingman 4 points 2 months ago

That’s fair! You can create an issue now with a branch in your repo as a proof of concept. Don’t wait to figure it out!

I am really curious tho and poking around myself.

[–] [email protected] 9 points 2 months ago

For those who, like me, have never heard of a .gitkeep file: https://stackoverflow.com/questions/7229885/what-are-the-differences-between-gitignore-and-gitkeep

[–] [email protected] 4 points 2 months ago (1 children)

Reminds me of this sort of story https://www.snopes.com/fact-check/know-where-man/

[–] [email protected] 1 points 2 months ago

Great analogy. I'm my experience there is an inverse relationship between the amount of time to find a bug and the size of the patch to fix it.

Sometimes it's just one character off.