Hi all!

As many of you have noticed, many Lemmy.World communities introduced a bot: @MediaBiasFactChecker@lemmy.world. This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.

The !news@lemmy.world mods want to give the community a chance to voice their thoughts on some potential changes to the MBFC bot. We have heard concerns that tend to fall into a few buckets. The most common concern we’ve heard is that the bot’s comment is too long. To address this, we’ve implemented a spoiler tag so that users need to click to see more information. We’ve also cut wording about donations that people argued made the bot feel like an ad.

Another common concern people have is with MBFC’s definition of “left” and “right,” which tend to be influenced by the American Overton window. Similarly, some have expressed that they feel MBFC’s process of rating reliability and credibility is opaque and/or subjective. To address this, we have discussed creating our own open source system of scoring news sources. We would essentially start with third-party ratings, including MBFC, and create an aggregate rating. We could also open a path for users to vote, so that any rating would reflect our instance’s opinions of a source. We would love to hear your thoughts on this, as well as suggestions for sources that rate news outlets’ bias, reliability, and/or credibility. Feel free to use this thread to share other constructive criticism about the bot too.

  • Aurenkin@sh.itjust.works
    link
    fedilink
    arrow-up
    4
    ·
    edit-2
    3 months ago

    My personal view is to remove the bot. I don’t think we should be promoting one organisations particular views as an authority. My suggestion would be to replace it with a pinned post linking to useful resources for critical thinking and analysing news. Teaching to fish vs giving a fish kind of thing.

    If we are determined to have a bot like this as a community then I would strongly suggest at the very least removing the bias rating. The factuality is based on an objective measure of failed fact checks which you can click through to see. Although this still has problems, sometimes corrections or retractions by the publisher are taken note of and sometimes not, leaving the reader with potentially a false impression of the reliability of the source.

    For the bias rating, however, it is completely subjective and sometimes the claimed reasons for the rating actually contradict themselves or other 3rd party analysis. I made a thread on this in the support community but TLDR, see if you can tell the specific reason for the BBC’s bias rating of left-centre. I personally can’t. Is it because they posted a negative sounding headline about Trump once or is it biased story selection? What does biased story selection mean and how is it measured? This is troubling because in my view it casts doubt on the reliability of the whole system.

    I can’t see how this can help advance the goal (and it is a good goal) of being aware of source bias when in effect, we are simply adding another bias to contend with. I suspect it’s actually an intractable problem which is why I suggest linking to educational resources instead. In my home country critical analysis of news is a required course but it’s probably not the case everywhere and honestly I could probably use a refresher myself if some good sources exist for that.

    Thanks for those involved in the bot though for their work and for being open to feedback. I think the goal is a good one, I just don’t think this solution really helps but I’m sure others have different views.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      3 months ago

      One issue with poor media literacy is that I don’t think people are going to go out of their way to improve their literacy on their own just from a pinned post. We could include a link in the bot’s comment to a resource like that though.

      Do you think that the bias rating would be improved by aggregating multiple factors checkers’ opinions into one score?

  • TheTechnician27@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    3 months ago

    To clarify what MBFC considers “MIXED” factual reporting (the same rating they give known disinformation factory Breitbart):

    Further, while The Guardian has failed several fact checks, they also produce an incredible amount of content; therefore, most stories are accurate, but the reader must beware, and hence why we assign them a Mixed rating for factual reporting.

    They list like five fact checks, while The Guardian puts out basically quintuple that every day. And moreover, this is the sort of asinine nitpick that they classify as a “fact check”.

    “Private renting is making people ill.” “Private renting is making people ill, but maybe this happens with other housing situations too, we don’t know, so we rate this as false.”

    MBFC’s ratings for “factual reporting” are a joke.

  • bloodfart@lemmy.ml
    link
    fedilink
    arrow-up
    1
    ·
    3 months ago

    You don’t need to manufacture an authoritative source of truth as you the mods see it.

    Just write down what you see as the truth and that you’ll ban anyone who speaks out against it.

    Stop trying to build a machine to do the work of creating an echo chamber for you.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      2
      ·
      3 months ago

      Always appreciate feedback! As a .ml user, I’d love to hear more about your thoughts on echo chambers

  • MimicJar@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    3 months ago

    Ban it and all bots honestly. I hate seeing a comment on a thread just to find out it’s a bot. If not use like this continues we might see a fresh post with 6 new comments, all of them bots that don’t add to the discussion.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      3 months ago

      Okay. This post is an attempt to solicit constructive criticism/feedback. Do you have anything more concrete to share?

      • I_Has_A_Hat@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        3 months ago

        Yes, maybe don’t have the bot be the first and only response on every single post. Let them gain the tiniest bit of traction first. It’s beyond annoying to see an article, go to the comments, and your bot be the only response.

  • Zonetrooper@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    3 months ago

    I’m frankly rather concerned about the idea of crowdsourcing or voting on “reliability”, because - let’s be honest here - Lemmy’s population can have highly skewed perspectives on what constitutes “accurate”, “unbiased”, or “reliable” reporting of events. I’m concerned that opening this to influence by users’ preconceived notions would result in a reinforced echo chamber, where only sources which already agree with their perspectives are listed as “accurate”. It’d effectively turning this into a bias bot rather than a bias fact checking bot.

    Aggregating from a number of rigorous, widely-accepted, and outside sources would seem to be a more suitable solution, although I can’t comment on how much programming it would take to produce an aggregate result. Perhaps just briefly listing results from a number of fact checkers?

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      3 months ago

      That’s fair. One idea could be a separate “community rating” and one that is more professional. Think Metacritic, RottenTomatoes, etc

  • Beetschnapps@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    3 months ago

    The bot is basically loud as fuck in a way that disrupts the comment feed.

    Imagine how comments should create and add to a conversation. Imagine how various lemmy clients feed or service that conversation….

    Now imagine how a double dropdown big as fuck post says “fuck you” to that conversation.

    Just please consider how the form of your shit can be just as imposing as the content, which I really appreciate.

    Yet somehow your posts always have me thinking “shut the fuck up” which seems antithetical to building a community.

  • anubis119@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    3 months ago

    I think this tool, while probably well-intended, only adds to the polarization problem of the world.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      3 months ago

      Can you elaborate? Like, do you think the bot would be better if it didn’t label things as “left” or “right” (ie: remove the bias rating) or do you think the reliability/credibility ratings have the same issue?

  • AlexWIWA@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    3 months ago

    Tell the bot to never be the first comment. I find it very frustrating when I see “a comment on this post” and it’s just the bot. I’m here to read what people have to say so it is very annoying when I think someone said something and it’s just the bot.

    There was even a front page meme about this last year, but with another noisy bot. Lemmy doesn’t bury new comments like Reddit does, so there’s no real penalty to making the bot wait.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      3 months ago

      Just to clarify, are you saying the bot should be triggered after someone else comments on a post?

  • qevlarr@lemmy.world
    link
    fedilink
    arrow-up
    1
    arrow-down
    1
    ·
    3 months ago

    Remove it.

    No need for a bot. Obvious misinformation should be removed by the mods. Bias is too subjective to be adjudicated by the mods. Just drop it already. It’s consistently downvoted into oblivion for a reason. The feedback has been petty damn obvious. This whole thread is just because the mods are so sure they’re right that they can’t listen to the feedback they already got. Just kill the bot.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      edit-2
      3 months ago

      I’ll be honest, that’s probably outside of the scope of what we can do for now. It’s definitely valuable feedback in general and I wish I could offer some kind of solution but that’s probably even outside the control of the instance admins.

      Someone can feel free to correct me if I’m wrong!

  • ZombiFrancis@sh.itjust.works
    link
    fedilink
    arrow-up
    0
    ·
    3 months ago

    I’m gonna be Left-Center on this with reliable credibility that the bot is useless at best.

    It is reporting on the source, not the content, of what is posted which is already going to be a problem for discourse.

    If there are media sources that are known or proven to be a problem, I would find it preferable the bot just alert that and ignore anything else.

    • jeffw@lemmy.worldOPM
      link
      fedilink
      arrow-up
      0
      arrow-down
      1
      ·
      3 months ago

      I appreciate the joke lol. But on a serious note, it sounds like you’re saying it’s not actually 100% useless, just that it’s being deployed too widely. Any specific suggestions on what the bot should say on those questionable sources?