I am interested in checking out the historical growth of a particular community. Lemmy Explorer crawls for data about the Lemmyverse every 24 hours or something, and that data is made available on their website. But I can only find where to download the latest data. Is there somewhere that I can find historical data? Does Lemmy Explorer archive these anywhere, or does it just overwrite the previous data each time it crawls?

  • Andrew@piefed.social
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    7 months ago

    Yeah, but the community started before the bot at !trendingcommunities@feddit.nl did, lemmy.world had a broken API for much of August (hence the jump), and the subscriber count never changes that much otherwise, but here’s what I have:

    day subs
    2023-07-24 70
    2023-07-26 71
    2023-07-28 75
    2023-08-03 76
    2023-08-05 77
    2023-09-03 86
    2023-09-04 87
    2023-09-05 88
    2023-09-11 89
    2023-09-29 90
    2023-10-08 89
    2023-10-14 90
    2023-10-15 91
    2023-10-18 92
    2023-10-21 93
    2023-10-22 94
    2023-11-12 93
    2023-11-17 96
    2023-11-19 97
    2023-11-20 96
    2023-12-15 97
    2024-01-05 98
    2024-01-10 99
    2024-03-08 100
    2024-03-10 101
    2024-03-11 102
    2024-03-15 103
    2024-03-22 104
    2024-04-04 105

    Incidentally, if you want to know how broken the Fediverse is right now, lemmy.ml is the only version that has both of your comments. piefed.social (my instance) has 1, lemmy.one (OP’s instance) had the other one, and lemmy.world (where this community is hosted) had none (edit: this was before I made this comment, which has forced a bit of a re-sync).

      • freamon@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        7 months ago

        Oh, right. The chat on GitHub is over my head, but I would have thought that solving the problem of instances sending every activity 2 or 3 times would help with that, since even rejecting something as a duplicate must eat up some time.

        • MV (Jerboa dev)@programming.dev
          link
          fedilink
          English
          arrow-up
          1
          ·
          7 months ago

          Yes but now it must wait for acknowledgement of a request (activity), before sending the next one. If one request takes 333ms means you can do max 3 requests per second. Now big instances like lemmy.world have activity above that so instances too far will perpetual lag behind