small changes in ‘tests/*’, "robustifying" against the
possibility of osVersion being NULL;
with thank to Michael Chirico for alerting me.
small changes in ‘tests/lmrob-psifns.R’ and ‘*.Rout.save’, notably
avoiding small diffs on "M1mac" and "noLD" (--disable-long-double).
lmrob.fit() gets new option bare.only for some
speed up, in case a relatively “bare” result is sufficient.
fix another LaTeX no more wanting math in section
titles (same problem as 0.99-3), now for ‘lmrob_simulation.Rnw’.
Now also use \texorpdfstring inside \title{} of
‘psi_functions.Rnw’ and use newtheorem{..}
after \hypersetup{...} (thanks to Kurt).
avoid some ggplot2 warnings (about omitting obs.)
do not warn about skipping equilibration for p=1.
define STRICT_R_HEADERS now (src/*.[ch]).
Again adapt ‘../vignettes/lmrob_simulation.Rnw’ to
ggplot2 changes ( s/ fun.y / fun / ).
‘tests/subsample.R’ now shows "Matrix" info (as that gave confusing error during Matrix / R-devel changes).
Latest LaTeX does not like math in \section{.}; hence
replacing \psi by \texorpdfstring{$\psi$}{psi}.
fix thinko bug introduced exact fit checking in 0.99-0; thanks to the report by Thomas Mang.
document the ... argument(s) of nlrob.control().
‘DESCRIPTION’: more URLSs.
‘*.Rd’ tweaks; avoiding other NOTEs.
‘src/lmrob.c,mc.c,monitor.c’: format %lld etc
In case it finds an exact fit, i.e., k > n/2
residuals are (practically) zero, lmrob.S() now returns that,
scale = 0, and robustness weights weights(., "robustness")
which are either 0 or 1 quickly and more consistently.
This is not yet satisfactorily working for the “large n” case.
**** TODO: Optionally (but off by default?? for back compatibility??),
lmrob() initialized by such an exact-fit estimator may use a
(user-specified or automatic) scale \hat{s} > 0 to
continue providing efficient SM / MM estimates.
lmrob.control() gets class "lmrobCtrl" which can be
print()ed and update()d.
New tuning const zero.scale = 1e-10 instead of hardwired
EPS_ZERO in lmrob.S()'s C code; additionally the exact
fit / zero-residuals checking is finally y-scale equivariant, using
1/n ||y||_1 = mean(abs(y)).
Fixed covMcd(): corrected the consistency correction
factor for the reweighted estimate: was MCDCons(p, sum(w)/n),
now is MCDCons(p, 0.975), see Croux and Haesbroeck (1999).
Consequently, often covMcd() and estimates based on it, such
as BYlogreg() or ltsReg() are slightly changed.
registered residuals.lmrob.S and changed
formals compatible with generic to (object, ...).
lmrob.S() in C level refine_fast_s() no longer
modifies beta_cand[] which notably fixes a bug found in
exact-fit examples previously resulting in coef() == 0
“results”, also for lmrob() which calls
lmrob.S() for its default initial estimate.
print(<lmrob.S>) finally works again as intended
(not printing a largish list); mostly by fixing internal
lmrob.control.minimal().
summary.lmrob(obj) now also gets a weights
component from obj if that is non-trivial; this may also
correct summary() output in weighted cases.
Minor fixes in plot.lts() related to par mfrow.
ltsReg() Help file changed - the note strongly advising NOT to use LTS regression removed.
argument mf e.g., in lmrob.fit() has been deprecated
since 2017-12-09 and is defunct now.
lmrob() tweaks, notably saying more for trace.lev >= 3.
replaceed (almost) all Calloc() by R_alloc() in C code;
e.g., prevents leakage from user interrupt.
Fixed covMcd(): raw.only=TRUE does not work in the case nsamp="deterministic"
New lmc() and rmc() for robust tail weight
(“robust kurtosis”).
Our ‘tests/*.R’ no longer rely on the Matrix ‘test-tools.R’ collection but rather use our much smaller own one in ‘xtraR/test-tools.R’.
fixed \ escape in ‘man/aircraft.Rd’.
initialize equed in ‘src/lmrob.c’ to avoid false positive compiler warning.
document "internal" .regularize.Mpsi().
replace () by (void) arguments for F77_SUB(println).
huberM(x, k) now also works in small sample cases with
+/-Inf, e.g. for x = c(-Inf, 1).
New huberize() utility, notably by default used in mc().
mc() now by default uses doScale=FALSE in order
to guarantee convergence also in extreme cases when used together
with new c.huberize = 1e11. Both are not back
compatible but clearly better in extreme cases, as assessed by
Lukas Graz' BSc thesis.
Note that the behavior of the new default doScale=FALSE has
been amended in the C code to be more compatible in its eps
checks with (the previous default) doScale=TRUE.
Lastly, a message() is printed currently, at most once per R session,
whenever mc() is called without an explicit specification of doScale=*,
alerting users to the changed default. This message can be turned
off by setting options(mc_doScale_quiet = TRUE).
New dataset data(x30o50) as an example of potentially
problematic data, not just for mc().
rather "truncating" +/-Inf to <large>, use
NA.OK = TRUE in Qn() and Sn()'s .C() call.
Our C code no longer uses the DOUBLE_* constants from S,
but rather standard <float.h>.
Qn(x) and Sn(x) gain an na.rm = FALSE
option and now work when x has NAs and contains
\pm Inf.
No longer include ‘Rdefines.h’ as it is somewhat deprecated.
In ‘src/lmrob.c’ add USE_FC_LEN_T and use FCONE
rather than the older FCLEN approach.
scaleTau2() gets new optional iter = 1 and
tol.iter arguments; mostly experimentally to see if or when
iteration makes sense.
Qn(x, *) gets new optional k = . to indicate the
“quantile” i.e., order statistic to be computed (with default as
previously hard-coded).
Experimentally to try for cases where more than n/2 observations
coincide (with the median), i.e., x[i] == x0 == median(x[]),
and hence Qn(x) and mad(x) are zero.
adjOutlyingness() gets new option IQRtype = 7.
For tests: *again* differences found in the non-sensical
adjOutlyingness() example (with large p/n, hence many "random" values
in the order of 1e15).
Disable the test for now (and record the result in *.Rout).
The test() utility in ‘tests/lmrob-ex12.R’ no
longer calls matrix(x, n,4) where the length of x does not
match 4n. Similar change in ‘tests/mc-strict.R’
Use \CRANpkg{.} in most places, providing web
links to the respective CRAN package page.
adjOutlyingness() now gains optional parameters to be
passed to mc().
update the internal man page, so new checkRdContents()
is happy.
fix several ‘\url{.}’'s that now are diagnosed as ‘moved’.
adjOutlyingness() finally works with p.samp > p.
scaleTau2() now works with Inf and very large
values, and obeys new na.rm = FALSE argument.
add check.environment=FALSE to some of the
all.equal() calls (for 'R-devel', i.e., future R 4.1.x).
wgt.himedian(numeric()) now returns NA instead of
occasionally seg.faulting or inf.looping. Ditto for a case when
called from Qn().
splitFrame() now treats character columns also as
categorical (the same as factors).
Small updates, also in checks for newer compiler settings,
e.g., FCLEN macro; also F77_*() etc, in order to fix 'LTO'
issues.
More careful or less calling intpr(): correct "Rank" of array
(for gfortran/gcc 10, when -fallow-argument-mismatch is not set).
For tests: try harder in moreSessionInfo() to detect
openBLAS (should work for KH's debian openBLAS setup).
R_CHECK_LENGTH_1* fix in default value for ask in
plot methods for mcd and lts
‘src/rowMedians.c’ gets better PROTECT()ion thanks to Tomas Kalibera's "rchk"s.
losen the strict tests more, convergence issues on all
non-standard BLAS platforms. Now modularized the partly platform
dependent testing into moreSessionInfo() in
‘./xtraR/platform-sessionInfo.R’. Further, for now use
RNGversion("3.5.0") e.g. in ‘../tests/mc-strict.R’.
losen the strict tests, notably in
‘tests/nlregrob-tst.R’ to work around nls()
convergence issues on all non-standard BLAS platforms.
Update lmrob(<empty multivariate>) to the change in R
(incl 3.5.1 patched).
mc() gets new optional doScale argument, and
increased defaults for the tolerances
eps1 = 1e-14, eps2 = 1e-15
such that it should converge by default in more cases.
A na.action is now kept in summary(lmrob(*)),
and when print()ing it, a note about omitted observations,
e.g., because of NA's, is made as for lm().
Internal lmrob.weights(): more "resistant" in case
scale=0, using na.rm=TRUE (report only, no reprex).
lmrob(*, trace.lev >= 2) now shows some information
about the number of find_scale() iterations used (as these
are now stored C internally).
‘src/robustbase.h’: is_redescender now is
static inline, needed for some compilers, e.g., on
ubuntu 18.04.
Fixing R-forge bug(s) 6588 (and 6590, 6593),
https://r-forge.r-project.org/tracker/index.php?func=detail&aid=6588&group_id=59&atid=302
The ‘Usage:’s in the data set help pages now say
data(<..>, package="robustbase").
The ‘lmrob_simulation’ vignette now should continue to work with upcoming package ggplot2.
Mpsi(x, c, psi="huber", deriv=-1) now gives rho(x)
instead of mostly Inf.
.psi.const(*, "lqq") now also gives a
"constants" attribute.
more examples and help on Mpsi() etc functions and tuning constants.
The S estimator lmrob.S() and M-S estimator now both
make use of the new lmrob.control() argument
scale.tol which defaults to 1e-10, its formerly
hardwired value.
lmrob.S() further gets a new option only.scale = FALSE,
which when true allows to only compute the S scale estimate. In
that case, but also generally, trace.lev = 3 or larger also
produces output showing the C level find_scale() iterations.
(By Manuel Koller) There's now a small C API to call our
Mpsi() etc from C code in other packages, as
C_psi(), etc; using new ‘../inst/include/robustbase.h’.
nlrob()$call$algorithm now always contains the
algorithm used as a character string, compatibly
with nls().
new data set steamUse.
Vignette ‘lmrob_simulation.Rnw’: fixed the wrong
“emprical power” plots; with faster ggplot2, remove
all eval=FALSE for plots and longer store the
‘*.pdf’s.
nlrob() gets model option to ask for the
model.frame to be returned.
lmrob(..., method = "S") no longer necessarily
produces a warning in .vcov.w().
nlrob() returns a correct dataClasses component.
For use in non-R-internal BLAS/Lapack libraries, several ‘tests/*.R’ examples have been tweaked.
fullRank() utility for adjOutlyingness:
adjOutlyingness(), gets new options p.samp and
trace.lev, and when it fails to find enough good
directions, now checks the rank and mentions fullRank() in
case the matrix is (QR-)rank deficient.
The "lmrob" method for vcov() gets optional
argument complete = TRUE, where only complete =
FALSE is back compatible.
improved (error) messages in singular case in
.vcov.avar1().
.psi.const() is exported as well, and help is
improved about using and setting non-default psi tuning constants.
loosened some regression test tolerances (for alternatives to BLAS) in ‘tests/(mc-strict|poisson-ex)’.
scaleTau2(x, *) gains an optional sigma0
argument and now returns zero instead of NaN when
sigma0 == 0.
vignette(psi_functions)
adjOutlyingness(..., maxit.mult = max(100, p)) new option,
allowing more iterations for finding “good” projection directions.
summary(nlrob(*, method = "CM")) now works.
lmrob..MM..fit() now works again when x and
y are specified, but neither method nor obj is.
Now provide an "lmrob" method for the standard R
generic function hatvalues(), and also export its lower
level workhorse .lmrob.hat() (formerly hidden
lmrob.leverages()), which now by default has
names(.).
.lmrob.hat() (formerly lmrob.leverages()) has
been corrected for the rank-deficient case.
classPC(m) now also works for a 1-column matrix.
Hidden print() methods print.summary.lmrob()
and print.lmrob.S() get a showAlgo = TRUE argument
which the user can set to FALSE in order to suppress printing of
the “Algorithmic parameters”.
import (remaining parts) from "base" packages.
summary(<nlrob>) now also prints a summary on the residuals.
summary(lmrob(.))'s variance-covariance matrix is now
called cov instead of cov.unscaled (because it
is scaled). Code which has been using vcov(<lmrob>)
or <lmrob> $ cov, or even <summary.lmrob> $ cov is not affected.
Started this ‘NEWS.Rd’ file, to eventually replace the ‘ChangeLog’
plot.lmrob() also identifies largest residuals as
plot.lm(). Also gets new argument panel, and
add.smooth=TRUE behavior.
adapt to the fact that R 3.3.0 will have its own
sigma() S3 generic.
setup for having message translations (volunteers sought!).
more careful in ‘../src/mc.c’ (valgrind, thanks to Brian)
add missing documentation, better examples for predict.lmrob
warn.limit.* checks in lmrob*()
The ‘Co-Median’ covComed() from Maria Anna,
tweaked by Valentin and modified considerably by Martin.
Also document (and export) r6pack() utility.
New smoothWgt() function — “Biweight on a
Stick” — to be used as wgtFUN option for covMcd()
or covComed().
New utility colMedians() and rowMedians, as we
use columnwise medians in so many places.
Tweaks to medcouple(), after detecting flaws – which
may be inherent and have not been removed.
Improved its documentation and the adjOutlyingness()
one, notably its “central” case.
covMcd() with new options (kmini,
nmini) now ok (sometimes wrong in 0.92-1).
The deterministic MCD, via covMcd(..., nsamp="deterministic").
adjOutlyingness(): reverse the defaults of
clower and cupper and fix an “eternal”
erronous \pm swap; see new note in ‘man/adjOutlyingness.Rd’.
nlrob() now works with indexed vector parameters.
new outlierStats() (Manuel).
got rid of Fortran compiler warnings about ancient style.
nlrob(*, weigths), fixing R-forge bug #5988.
covMcd() fix for “MAD = 0” case (new
exactfit code 3).
.
.
.
.
.
.
.