Kevinger's picture
Add SetFit model
bd06a6d verified
metadata
library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
datasets:
  - Kevinger/hub-report-dataset
metrics:
  - accuracy
widget:
  - text: >-
      A 16-acre property once home to the long-shuttered Foxborough State
      Hospital will soon provide housing for 141 low-income senior households.


      Walnut Street, an affordable housing project being developed by the
      Affordable Housing Services Collaborative and Onyx, will turn land that
      has been vacant for decades into much-needed affordable housing.


      “Housing is empowering. No matter our age, it is a comfort not to worry
      about whether we can afford a place,” Onyx CEO Chanda Smart said at a
      press conference Thursday. “Senior housing for the town of Foxborough
      means that seniors who worked and raised their families here in Foxborough
      still have the opportunity to remain here.”


      Foxborough State Hospital opened in 1889 as the Massachusetts Hospital for
      Dipsomaniacs and Inebriates for treatment of alcoholism, according to the
      National Park Service, and was later converted to a standard psychiatric
      hospital. It closed in 1975, and parts of the property have already been
      redeveloped over the years.


      The Foxborough Housing Authority first began working on the project back
      in 2011. The land was transferred to the agency from the state in 2017 to
      be used for affordable housing.


      Acting Town Manager Paige Duncan told MassLive that the town held a number
      of community meetings to decide what to build on the property.


      “It was controversial, but what came out was a clear support for senior
      housing,” she said. “We really tried to address the needs of the community
      and we came up with a project that was sensitive to the area. We didn’t
      want a big block of buildings that towered over the neighborhood.”


      After that, she said, there was overwhelming support for the project. The
      permits were filed in February and approved by April, an almost unheard-of
      timeline.


      The finished project will provide 141 new apartments for residents age 55
      and over. Of those, 35 will be reserved for people making 30% or less of
      the area median income, and 85 will be for those making 60% AMI.
      Foxborough residents will be given preference for 70% of the units.


      A second phase of the project once this one is complete will add
      approximately 60 more units.


      Greg Spiers, chairman of the Housing Authority, said the new senior
      housing was badly needed, noting there are about 5,500 elderly and
      disabled people on public housing waiting lists in Massachusetts.


      “With 195 of those on that list Foxborough residents, that 70% local
      preference for first-time rentals is one of our goals,” he said. “The need
      is so great for affordable housing in our area and the entire state.”


      Housing and Livable Communities Secretary Ed Augustus praised the town for
      its dedication to creating more affordable housing, even though more than
      10% of its total housing units qualify as affordable. The 10% threshold is
      the state requirement to stop projects being filed under Chapter 40B, a
      law which allows affordable housing developments to bypass certain local
      permitting requirements.


      “You know that that is just an arbitrary number, but the real needs are
      significantly more than that,” Augustus said. “We need more communities to
      take note of what Foxborough is doing.”


      Lt. Gov. Kim Driscoll said the project is a good example of the use of
      surplus state land for housing. Gov. Maura Healey’s housing bond bill
      filed in October included a proposed $30 million that would support
      similar projects to use underutilized state property for housing. Healey
      also issued an executive order requesting state agencies to conduct an
      audit of their property to find land any surplus land suitable for this
      purpose.


      “Converting state-owned land to another entity can be a little bit of a
      torturous pathway. We know that building all the resources you need takes
      time,” Driscoll said Thursday. “How do we leverage the cost of land, which
      is one of the reasons housing is so expensive, to build the type of
      housing we need, but do it in a shorter timetable? That’s what this
      (project) is all about.”


      The project has received more than $25 million in state and federal
      funding, including through American Rescue Plan Act rental funds and state
      and federal Low Income Housing Tax Credits. Work on the site has not yet
      started.
  - text: >-
      “I was on my co-op last year for, like, a straight year, so coming back to
      campus feels kind of nerve-wracking,” said Jasmine Rodriguez, 21. “But I
      feel more experienced than I did in my first year. I had a lot of anxiety
      in my first year, but now it’s been really chill.”


      As about a dozen Northeastern University students went around a conference
      table talking about their college experiences, voices were soft and
      answers halting, at least initially. Gradually, though, the students at
      this check-in meeting last fall began to open up and speak candidly about
      the challenges and adjustments of college life.


      Advertisement


      The students were Black, Latino, and Asian American and ranged from
      first-years to seniors, mostly from neighborhoods across Boston; the
      majority were the first generation of their family to attend college. Most
      were their high schools’ valedictorians  hardworking, smart students who
      excelled despite lacking the advantages of many peers.


      That’s where The Valedictorian Project came in.


      The Boston-based nonprofit was founded in 2020 in response to the Boston
      Globe’s award-winning 2019 investigative series, The Valedictorians
      Project, which found that the city’s best and brightest public school
      students often encounter major obstacles to their academic and
      professional goals. (The Globe is not involved with the organization.)


      The Valedictorian Project matches participating high school graduates with
      peer mentors close to their age and a senior mentor who is an experienced
      professional in their intended line of work. It also provides a $500
      stipend for books and other necessities, and supplemental support through
      partnerships with other organizations to help students navigate their new
      lives on campus and choose career paths.


      “Many of our mentors are first-gen college students themselves,” cofounder
      and executive director Amy McDermott said in an interview. “Many navigated
      very similar personal backgrounds to our mentees. I hear often in our
      mentor interviews, they want to be that person that they wish they had in
      navigating college.”


      Advertisement


      This academic year marks a milestone for the organization, as its first
      cohort of college freshmen are now seniors.


      McDermott said the organization began by inviting Boston valedictorians to
      participate in its first year, then added students from Lawrence in year
      two, Brockton and Worcester in 2022, and Chelsea last spring.


      Jasmine Rodriguez took part in a roundtable discussion at Northeastern
      University for students participating in The Valedictorian Project.
      Jonathan Wiggs/Globe Staff


      Mentor John Marley, 30, of Taunton, said the organization helps level the
      playing field for young people who don’t come from privileged backgrounds.


      “Students from wealthier families have always had these mentorship
      relationships, always had these connections, and those things are just
      unseen,” said Marley, an attorney whose family came to the United States
      from Jamaica when he was 5. “Unfairly or not ... it’s always advantaged a
      particular group and class of students over another. And I think they do a
      good job addressing that.”


      This academic year, The Valedictorian Project is supporting 140 students,
      of whom about three-quarters are first-generation college students and
      roughly 85 percent are people of color, according to McDermott. Besides
      Northeastern, students in the program attend Boston University, Harvard,
      MIT, Tufts, Brown, Yale, Stanford, and other colleges around the country,
      she said.


      As a student of color at an expensive private university, Rodriguez said,
      “You have to physically go out and try to find people that look like you.
      And I feel like for everyone else, it’s very easy. They find them in their
      classes. But it’s like, in my classes there’ll be like one other Black or
      Hispanic person.”


      Advertisement


      Rodriguez, a Dorchester native majoring in communications and sociology,
      recently spent a year as a social media co-op for an organization that
      supports domestic violence victims. She is drawn to work that will help
      others, she said, because she saw people in need in her neighborhood and
      her own family as she grew up.


      “I saw a lot of people that look like me struggle and go through a lot of
      things,” she said. “My mom is an immigrant.  We grew up on Section 8
      [housing assistance]; we grew up on food stamps and stuff like that.”


      Ciana Omnis participated in a Northeastern University roundtable
      discussion for students participating in The Valedictorian Project.
      Jonathan Wiggs/Globe Staff


      Ciana Omnis, 20, a third-year industrial engineering major who grew up in
      Florida, moved to Dorchester at age 14, and was the 2021 valedictorian at
      Brighton High School. She is the eldest of three children, so she can’t
      lean on older siblings for advice, she said.


      Her father, a truck driver who immigrated to the United States from Haiti,
      didn’t complete high school, she said, while her mother, a health care
      administrator, completed an associate’s degree but doesn’t yet have her
      bachelor’s.


      “I’ve met a lot of people in college who have parents who have done
      four-year degrees or whatnot, or even other kinds of higher education, so
      they’re able to get advice from their parents,” Omnis said. “For me, it’s
      been a bit harder, because I have to kind of figure out certain things on
      my own.”


      Advertisement


      Her mentors help fill that gap, she said, and the program helps her “meet
      other people who have the same background as me.”


      After they met through a Valedictorian Project event, John Le, who was the
      2022 valedictorian at East Boston High School, became friends with Connor
      Lashley, the 2022 valedictorian at Jeremiah E. Burke High School in
      Dorchester.


      “One of the issues is socializing, like making a friend group, because
      from my experience, from each class you kind of like meet people there,
      but if you’re not in the same major, you might not be able to maintain a
      relationship with them,” said Le, 20.


      The Valedictorian Project, he added, “has really been helpful to meet
      people at Northeastern and ... find people with similar interests.”


      Lashley, 19, said his mentors have helped him learn how to network with
      others in his field and steered him toward scholarship opportunities, and
      he can count on their support whenever he needs it.


      “They’re pretty much available the same day if stuff comes up,” he said.


      Connor Lashley (left) and John Le took part in a roundtable discussion at
      Northeastern University for students participating in The Valedictorian
      Project. Jonathan Wiggs/Globe Staff


      Jeremy C. Fox can be reached at jeremy.fox@globe.com. Follow him
      @jeremycfox.
  - text: >-
      LEVERETT — Dakin Humane Society announced Wednesday that it has sold its
      former animal shelter at 63 Montague Road in Leverett to Better Together
      Dog Rescue.


      The news release didn’t include a sales price for the 3,480-square-foot
      building on 5 acres of land.


      But records at the Franklin County Registry of Deeds show the sale was for
      $575,000.
  - text: >-
      Joan Acocella, a cultural critic whose elegant, erudite essays about dance
      and literature appeared in The New Yorker and The New York Review of Books
      for more than four decades, died on Sunday at her home in Manhattan. She
      was 78.


      Her son, Bartholomew Acocella, said the cause was cancer.


      Ms. Acocella (pronounced ack-ah-CHELL-uh) wrote deeply about dancers and
      choreographers, including Mikhail Baryshnikov, Suzanne Farrell and George
      Balanchine. She scrutinized the vicissitudes of the New York City Ballet
      as well as the feats of the ballroom-dancing pros and celebrity oafs of
      the popular TV series “Dancing With the Stars.”


      She was The New Yorker’s dance critic from 1998 to 2019 and freelanced for
      The Review for 33 years. Her final articles for The Review were a two-part
      commentary in May on the biography “Mr. B: George Balanchine’s 20th
      Century,” by Jennifer Homans, her successor as The New Yorker’s dance
      critic.


      “What she wrote for us,” Emily Greenhouse, the editor of The Review, said
      in an email, “was often mischievous and always delicious  on crotch shots
      and cuss words, on Neapolitan hand gestures and Isadora Duncan’s emphasis
      on the solar plexus.”
  - text: >-
      StreetsblogMASS relies on the generous support of readers like you. Help
      us meet our year-end fundraising goals – give today!


      Last week, the labor union that represents most Boston police officers
      ratified a new contract that will introduce a number of reforms 
      including one that will start allowing civilians to take unwanted traffic
      detail shifts at construction sites.


      Under the former contract, Boston Police officers were the only people
      allowed to direct traffic for events and at construction sites. And they
      got paid extremely handsomely to do so: Boston police working as flaggers
      take home $60 an hour.


      In spite of that lucrative pay, Boston has a lot of construction sites,
      and fewer and fewer people who want to wear a police uniform.


      Boston City Councilor Kendra Lara told StreetsblogMASS earlier this year
      that over 40 percent of requests for police details at construction sites
      were going unfilled.


      The new labor contract removes a key barrier to reforming this system. But
      there is still a city ordinance on the books that requires at least one
      Boston Police officer at every city construction site "to protect the
      safety and general welfare of the public and to preserve the free
      circulation of traffic."


      A press spokesperson for Mayor Michelle Wu told StreetsblogMASS last week
      that their office is aware of the ordinance and has "identified multiple
      legal paths to implementing the new collective bargaining agreement."


      Old rules created absurd delays for street projects


      Councilor Lara also told StreetsblogMASS that many privately-run
      construction sites will simply ignore the law and do their work without a
      flagger if nobody responds to their requests for a detail.


      But construction firms who are sticklers for the rules can end up waiting
      months before a cop shows up to let them get their work done.


      That's what happened earlier this year in Oak Square, where the MBTA
      waited a full year for a police detail to show up so that they could paint
      some new crosswalks on Washington Street in Oak Square.


      Neighbors report that those crosswalks finally got painted in August 
      after a year-long wait.


      New contract hikes pay, allows civilian flaggers


      For all these reasons, allowing civilian flaggers at construction sites
      had been one of the city's key points of negotiation for a new collective
      bargaining agreement with its police union.


      Police details will still be required at "high-priority" events and
      construction sites, which involve major streets, busy intersections, or
      major events that anticipate over 5,000 attendees. The new contract would
      also pay cops who work those high-priority details "the highest overtime
      rate of the most senior officer."


      At other worksites, such as those along quiet neighborhood streets, Boston
      Police would still get the right of first refusal to fill traffic details.
      But if no Boston Police are interested, the work can be offered to other
      non-BPD certified officers, including campus police and retired Boston
      cops. If people with those qualifications still aren't interested,
      construction contractors can then offer the job to civilian workers.


      The agreement further specifies that anyone directing traffic in those
      lower-priority sites will earn $60 per hour.


      The new agreement will also ban cops from double-booking their shifts,
      which allowed some to get paid twice for the same period of time when one
      detail ended early.


      Incredibly, the police department is still using a labor-intensive
      paper-based system to assign details in each police district. The new
      agreement will allow for a citywide electronic scheduling system.
pipeline_tag: text-classification
inference: false
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Kevinger/hub-report-dataset
          type: Kevinger/hub-report-dataset
          split: test
        metrics:
          - type: accuracy
            value: 0.6529242569511026
            name: Accuracy

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model trained on the Kevinger/hub-report-dataset dataset that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Evaluation

Metrics

Label Accuracy
all 0.6529

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Kevinger/setfit-hub-multilabel-example")
# Run inference
preds = model("LEVERETT — Dakin Humane Society announced Wednesday that it has sold its former animal shelter at 63 Montague Road in Leverett to Better Together Dog Rescue.

The news release didn’t include a sales price for the 3,480-square-foot building on 5 acres of land.

But records at the Franklin County Registry of Deeds show the sale was for $575,000.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 53 386.3906 2161

Training Hyperparameters

  • batch_size: (8, 8)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 75
  • body_learning_rate: (2e-05, 2e-05)
  • head_learning_rate: 2e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0008 1 0.1304 -
0.0417 50 0.1596 -
0.0833 100 0.132 -
0.125 150 0.0064 -
0.1667 200 0.0017 -
0.2083 250 0.0004 -
0.25 300 0.0001 -
0.2917 350 0.0002 -
0.3333 400 0.0003 -
0.375 450 0.0002 -
0.4167 500 0.0001 -
0.4583 550 0.0002 -
0.5 600 0.0002 -
0.5417 650 0.0002 -
0.5833 700 0.0001 -
0.625 750 0.0001 -
0.6667 800 0.0001 -
0.7083 850 0.0001 -
0.75 900 0.0 -
0.7917 950 0.0001 -
0.8333 1000 0.0001 -
0.875 1050 0.0001 -
0.9167 1100 0.0001 -
0.9583 1150 0.0 -
1.0 1200 0.0001 -

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 2.3.1
  • Transformers: 4.35.2
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.17.0
  • Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}